PyCurl获取http响应过程时间

安装

pip install pycurl

MacOS安装PyCurl需要

export PYCURL_SSL_LIBRARY=openssl
export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include 
pip install pycurl --compile --no-cache-dir 

否则使用时会报错 ImportError: pycurl: libcurl link-time ssl backends (secure-transport, openssl) do not include compile-time ssl backend (none/other)

示例代码

import pycurl


def get(url):
    c = pycurl.Curl()
    c.setopt(pycurl.URL, url)
    c.setopt(pycurl.WRITEFUNCTION, lambda bytes: len(bytes))  # 不输出响应内存
    c.perform()  # 发送请求

    print('状态码', c.getinfo(pycurl.HTTP_CODE))
    print('域名解析时间', c.getinfo(pycurl.NAMELOOKUP_TIME))
    print('远程服务器连接时间', c.getinfo(pycurl.CONNECT_TIME))
    print('连接上后到开始传输时的时间', c.getinfo(pycurl.PRETRANSFER_TIME))
    print('接收到第一个字节的时间', c.getinfo(pycurl.STARTTRANSFER_TIME))
    print('上一请求总的时间', c.getinfo(pycurl.TOTAL_TIME))
    print('如果存在转向的话，花费的时间', c.getinfo(pycurl.REDIRECT_TIME))
if __name__ == '__main__':
    get('https://github.com/hanzhichao')

运行后结果如下：

状态码 200
域名解析时间 0.004715
远程服务器连接时间 0.211106
连接上后到开始传输时的时间 0.507748
接收到第一个字节的时间 1.326099
上一请求总的时间 1.620445
如果存在转向的话，花费的时间 0.0

模拟httpstat

import os
import math
import pycurl

def http_stat(url, response_output_file='response.txt'):
    c = pycurl.Curl()
    c.setopt(pycurl.URL, url)
    c.setopt(pycurl.WRITEFUNCTION, lambda bytes: open(response_output_file, 'wb').write(bytes))  # 输出响应到文件
    c.setopt(pycurl.HEADERFUNCTION,  lambda line: print(line.decode().strip()))  # 输出响应头
    c.perform()  # 发送请求

    print(f'Body stored in: {os.path.abspath(response_output_file)}\n')

    namelookup = c.getinfo(pycurl.NAMELOOKUP_TIME) * 1000  # 改为毫秒
    connect = c.getinfo(pycurl.CONNECT_TIME) * 1000
    pretransfer = c.getinfo(pycurl.PRETRANSFER_TIME) * 1000
    starttransfer = c.getinfo(pycurl.STARTTRANSFER_TIME) * 1000
    total = c.getinfo(pycurl.TOTAL_TIME) * 1000

    header = ['DNS Lookup', 'TCP Connection', 'TLS Handshake', 'Server Processing', 'Content Transfer']
    row1 = [f'{math.ceil(namelookup)}ms',
            f'{math.ceil(connect - namelookup)}ms',
            f'{math.ceil(pretransfer - connect)}ms',
            f'{math.ceil(starttransfer - pretransfer)}ms',
            f'{math.ceil(total - starttransfer)}ms']
    row2 = [''] * 5
    row3 = [f'namelookup:{int(namelookup)}ms', '', '', '']
    row4 = ['', f'connect:{int(connect)}ms', '', '']
    row5 = ['', '', f'pretransfer:{int(pretransfer)}ms', '']
    row6 = ['', '', '', f'starttransfer:{int(starttransfer)}ms']
    row7 = ['', '', '', '', f'total:{int(total)}ms']
    print(' {:^20} {:^20} {:^20} {:^20} {:^20} '.format(*header))
    print('[{:^20}|{:^20}|{:^20}|{:^20}|{:^20}]'.format(*row1))
    print(' {:^20}|{:^20}|{:^20}|{:^20}|{:^20}|'.format(*row2))
    print(' {:^41}|{:^20}|{:^20}|{:^20}|'.format(*row3))
    print(' {:^20} {:^41}|{:^20}|{:^20}|'.format(*row4))
    print(' {:^20} {:^20} {:^41}|{:^20}|'.format(*row5))
    print(' {:^20} {:^20} {:^20} {:^41}|'.format(*row6))
    print(' {:^20} {:^20} {:^20} {:^20} {:^41}'.format(*row7))


http_stat('https://http2.pro/api/v1')

执行后输出如下：

HTTP/2 200
x-content-type-options: nosniff
strict-transport-security: max-age=31536000;includeSubDomains;preload
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
referrer-policy: same-origin
content-security-policy: form-action 'self';upgrade-insecure-requests;manifest-src 'self';
cache-control: max-age=2592000
expires: Thu, 10 Aug 2023 07:27:03 GMT
vary: Accept-Encoding
content-type: application/json; charset=utf-8
date: Tue, 11 Jul 2023 07:27:03 GMT
server: Apache

Body stored in: /Users/superhin/Projects/试验/day03/response.txt

      DNS Lookup         TCP Connection       TLS Handshake      Server Processing     Content Transfer   
[        6ms         |       202ms        |       715ms        |       276ms        |        1ms         ]
                     |                    |                    |                    |                    |
              namelookup:5ms              |                    |                    |                    |
                                    connect:206ms              |                    |                    |
                                                       pretransfer:920ms            |                    |
                                                                          starttransfer:1196ms           |
                                                                                                   total:1196ms   

PyCurl 的一些响应信息

<参考： http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html>

• pycurl.NAMELOOKUP_TIME：域名解析时间
• pycurl.CONNECT_TIME：远程服务器连接时间
• pycurl.PRETRANSFER_TIME：连接上后到开始传输时的时间
• pycurl.STARTTRANSFER_TIME：接收到第一个字节的时间
• pycurl.TOTAL_TIME：上一请求总的时间
• pycurl.REDIRECT_TIME：如果存在转向的话，花费的时间
• pycurl.EFFECTIVE_URL
• pycurl.HTTP_CODE HTT： 响应代码
• pycurl.REDIRECT_COUNT：重定向的次数
• pycurl.SIZE_UPLOAD：上传的数据大小
• pycurl.SIZE_DOWNLOAD：下载的数据大小
• pycurl.SPEED_UPLOAD：上传速度
• pycurl.HEADER_SIZE：头部大小
• pycurl.REQUEST_SIZE：请求大小
• pycurl.CONTENT_LENGTH_DOWNLOAD：下载内容长度
• pycurl.CONTENT_LENGTH_UPLOAD：上传内容长度
• pycurl.CONTENT_TYPE：内容的类型
• pycurl.RESPONSE_CODE：响应代码
• pycurl.SPEED_DOWNLOAD：下载速度
• pycurl.SSL_VERIFYRESULT
• pycurl.INFO_FILETIME ：文件的时间信息
• pycurl.HTTP_CONNECTCODE HTTP：连接代码
• pycurl.HTTPAUTH_AVAIL
• pycurl.PROXYAUTH_AVAIL
• pycurl.OS_ERRNO
• pycurl.NUM_CONNECTS
• pycurl.SSL_ENGINES
• pycurl.INFO_COOKIELIST
• pycurl.LASTSOCKET
• pycurl.FTP_ENTRY_PATH