爬虫开始

1.安装scrapy

pip install scrapy

 

2.新建scrapy项目

scrapy startproject douban 

 

3.新建scrapy_spider

cd douban/spiders 

scrapy genspider douban_spider movie.douban.com

 

4.运行项目并保持到csv或json

 

# 运行程序
scrapy crawl douban_spider

# 保存文件到json
scrapy crawl douban_spider -o test.json
 
# 保持文件到csv

scrapy crawl douban_spider -o test.csv