爬虫 -- beautifulSoup

参考：https://www.cnblogs.com/wupeiqI/articles/6283017.html

　　 http://cn.python-requests.org/zh_CN/latest/user/quickstart.html#id4

概述

　　BeautifulSoup是一个模块，该模块用于接收一个HTML或XML字符串，然后将其进行格式化，之后遍可以使用他提供的方法进行快速查找指定元素，过滤提取数据。从而使得在HTML或XML中查找指定元素变得简单。

安装

pip3 install beautifulsoup4

使用示例1

from bs4 import BeautifulSoup
 
html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
    ...
</body>
</html>
"""
 
soup = BeautifulSoup(html_doc, features="lxml")

使用示例2

from bs4 import BeautifulSoup
 
html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
asdf
    <div class="title">
        <b>The Dormouse's story总共</b>
        <h1>f</h1>
    </div>
<div class="story">Once upon a time there were three little sisters; and their names were
    <a  class="sister0" id="link1">Els<span>f</span>ie</a>,
    <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
    <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</div>
ad<br/>sf
<p class="story">...</p>
</body>
</html>
"""
 
soup = BeautifulSoup(html_doc, features="lxml")
# 找到第一个a标签
tag1 = soup.find(name='a')
# 找到所有的a标签
tag2 = soup.find_all(name='a')
# 找到id＝link2的标签
tag3 = soup.select('#link2')

简单API

　　name，标签名称

# tag = soup.find('a')
# name = tag.name # 获取
# print(name)
# tag.name = 'span' # 设置
# print(soup)

　　attr，标签属性

# tag = soup.find('a')
# attrs = tag.attrs    # 获取
# print(attrs)
# tag.attrs = {'ik':123} # 设置
# tag.attrs['id'] = 'iiiii' # 设置
# print(soup)

　　children，所有子标签

# body = soup.find('body')
# v = body.children

　　children，所有子子孙孙标签

# body = soup.find('body')
# v = body.descendants

　　clear,将标签的所有子标签全部清空（保留标签名）

# tag = soup.find('body')
# tag.clear()
# print(soup)

posted @ 2019-09-06 15:50 MISS&MISS 阅读(67) 评论(0) 收藏举报

刷新页面返回顶部