python采集连续性网页的标题title

如果网页有连续性比如:baidu.com/1.html......baidu.com/10001.html那么就可以使用本python文件进行采集。

import requests
from bs4 import BeautifulSoup
s=1#网页开始的参数
e=10001#网页结束的参数 for _ in e:
  url = f"http://example.com/{_}.html"
  response = requests.get(url)

  soup = BeautifulSoup(response.text, "html.parser")
  title = soup.title.string

  print(title)
  

  

posted @ 2023-03-14 09:39  无恙大势  阅读(33)  评论(0)    收藏  举报