scrapy数据存入mongodb

存入mongodb的pipelines文件是这样子写的

from openpyxl import Workbook
from scrapy.conf import settings
import pymongo
class UsergentPipeline(object):
    def __init__(self):
        host=settings['MONGODB_HOST']
        port=settings['MONGODB_PORT']
        dbname=settings['MONGODB_DBNAME']#数据库名
        client=pymongo.MongoClient(host=host,port=port)
        tdb=client[dbname]
        self.port=tdb[settings['MONGODB_DOCNAME']]#表名
    def process_item(self, item, spider):
        agentinfo=dict(item)
        self.port.insert(agentinfo)
        return item

settings文件应该这样子写:

MONGODB_HOST = '192.168.90.9'
MONGODB_PORT = 27017
MONGODB_DBNAME = 'Flight'
MONGODB_DOCNAME = 't_user_gent'

 

posted @ 2016-10-10 09:32  你若精彩,蝴蝶自来  阅读(1641)  评论(0)    收藏  举报