爬虫项目:爬取汽车之家新闻资讯

_____egon新书python全套来袭https://egonlin.com/book.html

# requests+Beautifulsoup爬取汽车之家新闻

import requests
from bs4 import BeautifulSoup

response=requests.get('https://www.autohome.com.cn/news/')
response.encoding='gbk'

with open('a.html','w',encoding='utf-8') as f:
    f.write(response.text)
soup=BeautifulSoup(response.text,'lxml')


news=soup.find(id='auto-channel-lazyload-article').select('ul li a')


for tag in news:
    link=tag.attrs['href']
    imag=tag.select('.article-pic img')[0].attrs['src']
    title=tag.find('h3').get_text()
    sub_time=tag.find(class_='fn-left').get_text()
    browsing_num=tag.select('.fn-right em')[0].get_text()
    comment=tag.find('p').get_text()
    msg='''
    ======================================
    链接:http:%s
    图片:http:%s
    标题:%s
    发布时间:%s
    浏览数:%s
    介绍:%s
    ''' %(link,imag,title,sub_time,browsing_num,comment)

    print(msg)
 

 

posted @ 2017-11-06 05:13  linhaifeng  阅读(1040)  评论(0编辑  收藏  举报