Python之requests详解

Requests 是用Python语言编写，基于 urllib，采用 Apache2 Licensed 开源协议的 HTTP 库。它比 urllib 更加方便，可以节约我们大量的工作，完全满足 HTTP 测试需求。Requests 的哲学是以 PEP 20 的习语为中心开发的，所以它比 urllib 更加 Pythoner。更重要的一点是它支持 Python3 哦！

Beautiful is better than ugly.(美丽优于丑陋)
Explicit is better than implicit.(清楚优于含糊)
Simple is better than complex.(简单优于复杂)
Complex is better than complicated.(复杂优于繁琐)
Readability counts.(重要的是可读性)

安装 Requests

通过pip安装

pip install requests

或者，下载代码后安装：

$ git clone git://github.com/kennethreitz/requests.git
$ cd requests
$ python setup.py install

再懒一点，通过IDE安装吧，如pycharm！

请求

基本GET请求

import requests
response = requests.get("http://httpbin.org/get")
print(response.text)

带参数GET请求

import requests
response = requests.get("http://httpbin.org/get?name='abc'&age=23")
print(response.text)

import requests
data = {
    'name':'abc',
    'age': 23
}
response = requests.get("http://httpbin.org/get",params=data)
print(response.text)

添加headers

import requests
headers = {
    'User-Agent':'Mozilla/5.0(Windows NT 10.0; Win64; x64; rv:62.0)Gecko/20100101 Firefox/62.0'
}
response = requests.get('https://www.zhihu.com/explore',headers=headers)
print(response.status_code)

各种请求方式

requests.get(‘https://github.com/timeline.json’) #GET请求
requests.post(“http://httpbin.org/post”) #POST请求
requests.put(“http://httpbin.org/put”) #PUT请求
requests.delete(“http://httpbin.org/delete”) #DELETE请求
requests.head(“http://httpbin.org/get”) #HEAD请求
requests.options(“http://httpbin.org/get”) #OPTIONS请求

基本POST请求

import requests

headers = {
    'User-Agent':'Mozilla/5.0(Windows NT 10.0; Win64; x64; rv:62.0)Gecko/20100101 Firefox/62.0'
}
data = {'name':'张三','age':'22'}
response = requests.post("http://httpbin.org/post",data=data,headers=headers)
print(response.text)

POST发送JSON数据：

import requests,json

response = requests.post('https://api.github.com/some/endpoint', data=json.dumps({'some': 'data'}))
print(response.json())

文件上传

import requests

url = 'http://127.0.0.1:5000/upload'
files = {'file': open('01.jpg', 'rb')}
# files = {'file': ('01.jpg', open('01.jpg', 'rb'))}     #显式的设置文件名
response = requests.post(url, files=files)
print(response.text)

更加方便的是，你可以把字符串当着文件进行上传：

import requests

url = 'http://127.0.0.1:5000/upload'
files = {'file': ('test.txt', b'Hello Requests.')}  # 必需显式的设置文件名
response = requests.post(url, files=files)
print(response.text)

响应

response属性

response = requests.get('http://www.baidu.com')
print(type(response.status_code),response.status_code)
print(type(response.headers),response.headers)
print(type(response.cookies),response.cookies)
print(type(response.url),response.url)
print(type(response.history),response.history)

其他响应：

response.status_code #响应状态码
response.raw #返回原始响应体，也就是 urllib 的 response 对象，使用 r.raw.read() 读取
response.content #字节方式的响应体，会自动为你解码 gzip 和 deflate 压缩
response.text #字符串方式的响应体，会自动根据响应头部的字符编码进行解码
response.headers #以字典对象存储服务器响应头，但是这个字典比较特殊，字典键不区分大小写，若键不存在则返回None
#*特殊方法*#
response.json() #Requests中内置的JSON解码器
response.request.headers #返回请求头
response.raise_for_status() #失败请求(非200响应)抛出异常

获取请求头和响应头

import requests

response = requests.get('http://www.zhidaow.com')
print(response.headers['Content-Type'])
print(response.headers.get('content-type'))

print(response.request.headers)
print(response.request.headers['User-Agent'])

解析json

import requests,json
response = requests.get("http://httpbin.org/get")
print(response.json())
print(json.loads(response.text))
print(type(response.text),type(response.json()))

解析二进制文件

import requests,json
response = requests.get("https://github.com/favicom.ico")
print(type(response.text),type(response.content))
print(response.text)
print(response.content)
with open('favicom.ico','wb') as f:
    f.write(response.content)

高级操作

获取cookie

import requests

response = requests.get('http://www.baidu.com')
print(response.cookies)
print(response.cookies['content-type'])
for key,value in response.cookies:
    print(key + "=" + value)

要想发送你的cookies到服务器，可以使用 cookies 参数：

import requests
 
url = 'http://httpbin.org/cookies'
cookies = {'testCookies_1': 'Hello_Python3', 'testCookies_2': 'Hello_Requests'}
# 在Cookie Version 0中规定空格、方括号、圆括号、等于号、逗号、双引号、斜杠、问号、@，冒号，分号等特殊符号都不能作为Cookie的内容。
response = requests.get(url, cookies=cookies)
print(r.json())

会话维持(常用于模拟登录)

相当与两个浏览器发送请求，cookie不能共享

import requests

requests.get("http://httpbin.org/cookies/set/number/232423")
response = requests.get('http://httpbin.org/cookies')
print(response.text)

import requests

session = requests.Session()
session.get("http://httpbin.org/cookies/set/number/232423")
response = session.get('http://httpbin.org/cookies')
print(response.text)

证书验证

12306网站，使用的不是官方的证书验证，会导致SSL错误

import requests

response = requests.get('https://www.12306.cn')
print(response.status_code)

urllib3.disable_warnings()：关闭警告

verify=False:关闭证书认证

import requests
from  requests.packages import urllib3
urllib3.disable_warnings()
response = requests.get('https://www.12306.cn',verify=False)
print(response.status_code)

指定本地证书进行认证

import requests
response = requests.get('https://www.12306.cn',cart=('/path/server.crt','/path/key'))
print(response.status_code)

代理设置

import requests
proxies = {
    "http":'http://127.0.0.1:5465',
    "https":'https://127.0.0.1:5465',
}
response = requests.get('https://www.taobao.com',proxies=proxies)
print(response.status_code)

当代理需要认证时

import requests
proxies = {
    "http":'http://user:password@127.0.0.1:5465',
}
response = requests.get('https://www.taobao.com',proxies=proxies)
print(response.status_code)

socks代理

#pip3 install "requests[socks]"
import requests
proxies = {
    "http":'socks5://127.0.0.1:5465',
    "https":'socks5://127.0.0.1:5465',
}
response = requests.get('https://www.taobao.com',proxies=proxies)
print(response.status_code)

超时设置

import requests
from requests.exceptions import ReadTimeout,ConnectTimeout
try:
    response = requests.get('https://httpbin.org/get',timeout=0.5)
    print(response.status_code)
except ReadTimeout:
    print("Read Timeout")
except ConnectTimeout:
    print("Connect Timeout")

认证设置

import requests
from requests.auth import HTTPBasicAuth
 
r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd', auth=HTTPBasicAuth('user', 'passwd'))
# r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd', auth=('user', 'passwd'))    # 简写
print(r.json())

另一种非常流行的HTTP身份认证形式是摘要式身份认证，Requests对它的支持也是开箱即可用的:

requests.get(URL, auth=HTTPDigestAuth('user', 'pass'))

后记

1、官方文档
requests的具体安装过程请看：http://docs.python-requests.org/en/latest/user/install.html#install
requests的官方指南文档：http://docs.python-requests.org/en/latest/user/quickstart.html
requests的高级指南文档：http://docs.python-requests.org/en/latest/user/advanced.html#advanced

posted @ 2018-09-05 19:38 逐梦客！阅读(1408) 评论(0) 收藏举报

刷新页面返回顶部