一个完整的大作业

1.选一个自己感兴趣的主题。网址是:http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97,搜海南的游记。

 

2.网络上爬取相关的数据。

import requests
from bs4 import BeautifulSoup
import re
url='http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97' res
=requests.get(url) res.encoding='utf-8' soup=BeautifulSoup(res.text,'html.parser') for trips in soup.select('ul'): if len(trips.select('.title'))>0: title=trips.select('.title')[0].text url=trips.select('a')[0]['href'] print(title,url)

 

3.进行文本分析,生成词云。

import jieba
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import requests
from bs4 import BeautifulSoup
import re

youji=open("/Users/游记.txt","r",encoding='gb2312').read()
words=jieba.lcut(youji)
counts={}
ls=[]
for word in words:
    ls.append(word)
    if len(word)==1:
        continue
    else:
        counts[word]=counts.get(word,0)+1
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    word,count=items[i]
    print("{0:<10}{1:>5}".format(word,count))
    
wordlist=jieba.cut(youji, cut_all=True)
wl_split="/".join(wordlist)
cy=WordCloud(font_path=r'C:\Users\Administrator\Desktop\simheittf\simhei.ttf').generate(wl_split)
plt.imshow(cy)
plt.axis("off")
plt.show()

 

从词云可以看出,人们去海南旅游大都去的是三亚,海鲜、美景、享受等是那里的关键词。

 

posted on 2017-11-02 17:14  46陈远钊  阅读(264)  评论(0编辑  收藏  举报