Python编程快速上手-从Web抓取信息

利用webbrowser模块

 　　webbrowser模块的open()函数可以启动一个新浏览器

>>> import webbrowser
>>> webbrowser.open('http://www.baidu.com/')
True

 

用requests模块从Web下载文件

用requests.get() 函数下载一个网页

>>> import requests
>>> res = requests.get('http://www.gutenberg.org/cache/epub/1112/pg1112.txt')
>>> type(res)
<class 'requests.models.Response'>
>>> len(res.text)
179380
>>> print(res.text[:250])
The Project Gutenberg EBook of Romeo and Juliet, by William Shakespeare


*******************************************************************
THIS EBOOK WAS ONE OF PROJECT GUTENBERG'S EARLY FILES PRODUCED AT A
TIME WHEN PROOFING METHODS AND TOO

检查错误

>>> res = requests.get('http://inventwithpython.com/page_that_does_not_exist')
>>> res.raise_for_status()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python37\lib\site-packages\requests\models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://inventwithpython.com/page_that_does_not_exist
>>>

　　可以用try 和 except 语句将raise_for_status() 代码行包裹，处理这个错误，不让程序崩溃

import requests
res = requests.get('http://inventwithpython.com/page_that_does_not_exist')
try:
  res.raise_for_status()
except Exception as exc:
  print('There was a problem: %s' % (exc))

将下载的文件保存在磁盘

>>> import requests
>>> res = requests.get('http://www.gutenberg.org/cache/epub/1112/pg1112.txt')
>>> res.raise_for_status()
>>> playFile = open('RomeoAndJuliet.txt', 'wb')
>>> for chunk in res.iter_content(1000000):
...   playFile.write(chunk)
...
179382
>>>
>>> playFile.close()
>>>
>>>