Scrapy 中 Downloader 设置代理

from scrapy.downloadermiddlewares.httpproxy import HttpProxyMiddleware
class MyProxyMiddleware:
    def process_request(self, request, spider):
        # request.meta['proxy'] ='http://ip:port'
        # request.meta['proxy'] ='http://name:pwd@ip:port'
        request.meta['proxy'] ='http://139.224.211.212:8080'

Scrapy 中 Downloader 设置代理

DOWNLOADER_MIDDLEWARES = {
  'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware':None,
  "scrapy06.middlewares.MyProxyMiddleware": 543,
}

 

posted @ 2023-06-25 19:30  jiang_jiayun  阅读(45)  评论(0)    收藏  举报