Python-Scrapy shell 带头部headers请求

scrapy shell -s USER_AGENT="" request_url 就可以完成带头部的请求添加，如请求简书（不带头部请求时403错误）

scrapy shell -s USER_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" https://www.jianshu.com/recommendations/users?page=1

 

posted @ 2020-04-03 16:38  BruceLong  阅读(736)  评论(0)    收藏  举报