scrapy log设置和输出

scrapy log设置和输出

在settings.py 中设置log级别

LOG_LEVEL = 'WARNING'

log 级别

• CRITICAL - 严重错误(critical)
• ERROR - 一般错误(regular errors)
• WARNING - 警告信息(warning messages)
• INFO - 一般信息(informational messages)
• DEBUG - 调试信息(debugging messages)

在爬虫文件中添加代码

import sys
import scrapy
import logging


class QuotesSpider(scrapy.Spider):
    name = "taobao"
    start_urls = [
        'https://ai.taobao.com/',
    ]
    logger = logging.getLogger(__name__)

    def parse(self, response):
        links = response.xpath('//div[@class="chanel-tags clearfix"]//a')
        self.logger.warning(links)