一个完整的大作业

1.选一个自己感兴趣的主题。网址是：http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97，搜海南的游记。

 

2.网络上爬取相关的数据。

import requests
from bs4 import BeautifulSoup
import re
url='http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97'
res=requests.get(url)
res.encoding='utf-8'
soup=BeautifulSoup(res.text,'html.parser')

for trips in soup.select('ul'):
    if len(trips.select('.title'))>0:
      title=trips.select('.title')[0].text
      url=trips.select('a')[0]['href']
      
      print(title,url)

 

3.进行文本分析，生成词云。

import jieba
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import requests
from bs4 import BeautifulSoup
import re

youji=open("/Users/游记.txt","r",encoding='gb2312').read()
words=jieba.lcut(youji)
counts={}
ls=[]
for word in words:
    ls.append(word)
    if len(word)==1:
        continue
    else:
        counts[word]=counts.get(word,0)+1
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    word,count=items[i]
    print("{0:<10}{1:>5}".format(word,count))
    
wordlist=jieba.cut(youji, cut_all=True)
wl_split="/".join(wordlist)
cy=WordCloud(font_path=r'C:\Users\Administrator\Desktop\simheittf\simhei.ttf').generate(wl_split)
plt.imshow(cy)
plt.axis("off")
plt.show()

 

从词云可以看出，人们去海南旅游大都去的是三亚，海鲜、美景、享受等是那里的关键词。