随笔分类 - scrapy

scrapy

基础

摘要：1 安装scrapy 2 pip install wheel 3 下载Twisted https://www.lfd.uci.edu/~gohlke/pythonlibs/ 4 pip install Twisted-xxxxxx 5 pip install scrapy 创建工程 scrapy s 阅读全文

posted @ 2022-06-12 11:19 fbhell 阅读(21) 评论(0) 推荐(0)

settings.py

摘要：1 ROBOTSTXT_OBEY = False 2 3 LOG_LEVEL="ERROR" 4 5 USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86 阅读全文

posted @ 2022-06-12 11:19 fbhell 阅读(29) 评论(0) 推荐(0)