随笔分类 -  爬虫

设备模拟
摘要：/** * Copyright 2017 Google Inc. All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file 阅读全文
posted @ 2018-10-07 16:23 FromScratch 阅读(193) 评论(0) 推荐(0)
设置代理
摘要：/** * Copyright 2017 Google Inc. All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file 阅读全文
posted @ 2018-10-07 16:21 FromScratch 阅读(407) 评论(0) 推荐(0)
定义页面加载和导航时要执行的函数/自定义事件
摘要：http://www.cnblogs.com/dolphinX/p/7715268.html 好文/** * Copyright 2017 Google Inc. All rights reserved. * * Licensed under the Apache License, Version 阅读全文
posted @ 2018-10-07 16:11 FromScratch 阅读(385) 评论(0) 推荐(0)
搜索await page.waitForSelector(allResultsSelector);
摘要：/** * Copyright 2017 Google Inc. All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file 阅读全文
posted @ 2018-10-07 12:22 FromScratch 阅读(3450) 评论(0) 推荐(0)
默认浏览器设置
摘要：Default runtime settings 1. Uses Headless mode Puppeteer launches Chromium in headless mode. To launch a full version of Chromium, set the 'headless' 阅读全文
posted @ 2018-10-06 16:57 FromScratch 阅读(457) 评论(0) 推荐(0)
预备JS执行环境，预执行脚本
摘要：page.evaluateOnNewDocument(pageFunction, ...args) pageFunction <function|string> Function to be evaluated in browser context ...args <...Serializable> 阅读全文
posted @ 2018-10-06 16:45 FromScratch 阅读(282) 评论(0) 推荐(0)
在文档上下文里执行脚本，并返回结果
摘要：/在文档上下文里执行脚本，并返回结果(async () => { const browser = await puppeteer.launch({ args: ['--no-sandbox'], timeout: 10000, }); const page = await browser.newPa 阅读全文
posted @ 2018-10-06 15:32 FromScratch 阅读(236) 评论(0) 推荐(0)
拦截请求
摘要：const puppeteer = require('puppeteer'); puppeteer.launch().then(async browser => { const page = await browser.newPage(); //设置启用拦截 await page.setReques 阅读全文
posted @ 2018-10-06 10:15 FromScratch 阅读(616) 评论(0) 推荐(0)