curl_cffi 异步完整用法（AsyncSession）

curl_cffi 提供原生异步类 AsyncSession，基于 asyncio，性能远高于同步 Session，完美模拟浏览器指纹，解决 JS 检测、TLS 指纹、ja3 指纹拦截问题。

一、安装

pip install curl-cffi

 

二、基础异步模板（最小可运行）

import asyncio
from curl_cffi import requests

async def demo():
    # 创建异步会话，指定浏览器指纹（模拟真实浏览器）
    async with requests.AsyncSession(
        impersonate="chrome124",  # 指纹版本：chrome/firefox/safari
        timeout=30
    ) as session:
        # GET 请求
        resp = await session.get("https://httpbin.org/get")
        print(resp.status_code)
        print(resp.json())

        # POST 表单
        data = {"name": "test"}
        resp2 = await session.post("https://httpbin.org/post", data=data)
        print(resp2.text)

if __name__ == "__main__":
    asyncio.run(demo())

 

三、核心参数详解

1. 指纹 impersonate 常用值

• chrome120 / chrome124 / chrome130
• firefox117 / firefox128
• safari_17_0

 

# 运行查看所有支持指纹
from curl_cffi.const import IMPERSONATE_BROWSERS
print(IMPERSONATE_BROWSERS)

 

2. 请求常用参数（同步 / 异步通用）

await session.get(
    url="xxx",
    params={"a": 1},          # url参数
    headers={"User-Agent": "xxx"},
    cookies={"token": "xxx"},
    data={"k": "v"},          # x-www-form-urlencoded
    json={"key": "val"},      # application/json
    files={"file": open("a.txt", "rb")}, # 上传文件
    proxies={"http": "http://127.0.0.1:7890", "https": "http://127.0.0.1:7890"},
    allow_redirects=True,
    timeout=15,
)

 

四、并发大量请求（异步核心优势）

使用 asyncio.gather 批量并发，无需手动管理线程：

import asyncio
from curl_cffi import requests

async def fetch(session, url):
    resp = await session.get(url)
    return url, resp.status_code

async def batch_request():
    urls = [
        "https://httpbin.org/get",
        "https://httpbin.org/headers",
        "https://httpbin.org/ip"
    ]
    async with requests.AsyncSession(impersonate="chrome124") as session:
        # 构建任务列表
        tasks = [fetch(session, url) for url in urls]
        # 并发执行
        results = await asyncio.gather(*tasks)
        for url, code in results:
            print(f"{url} -> {code}")

if __name__ == "__main__":
    asyncio.run(batch_request())

 

五、持久 Cookie / 会话复用

AsyncSession 会自动保存同域 cookie，多次请求携带登录态：

async def login_and_fetch():
    async with requests.AsyncSession(impersonate="chrome124") as session:
        # 登录
        await session.post("https://httpbin.org/post", data={"user": "admin", "pwd": "123"})
        # 自动带上登录cookie
        resp = await session.get("https://httpbin.org/cookies")
        print(resp.json())

 

六、文件下载异步（流式）

async def download_file():
    url = "https://httpbin.org/image/png"
    async with requests.AsyncSession(impersonate="chrome124") as session:
        resp = await session.get(url, stream=True)
        with open("test.png", "wb") as f:
            async for chunk in resp.iter_content(chunk_size=8192):
                f.write(chunk)

 

七、代理配置

http/socks5 代理

proxies = {
    "http": "socks5://127.0.0.1:1080",
    "https": "socks5://127.0.0.1:1080"
}
await session.get(url, proxies=proxies)

 

八、关闭 SSL 验证（测试环境慎用）

await session.get(url, verify=False)

 

九、异常捕获

from curl_cffi.errors import CurlError, RequestError

async def safe_request(session, url):
    try:
        resp = await session.get(url, timeout=10)
        resp.raise_for_status()  # 4xx/5xx抛异常
        return resp
    except CurlError as e:
        print(f"Curl底层错误: {e}")
    except RequestError as e:
        print(f"请求异常: {e}")
    except Exception as e:
        print(f"其他错误: {e}")

 

十、不使用 async with 手动释放（不推荐）

session = requests.AsyncSession(impersonate="chrome124")
resp = await session.get("https://httpbin.org")
await session.close()  # 必须手动关闭释放资源

 

十一、与 aiohttp 对比优势

1. 自带完整浏览器 TLS/JA3 指纹，反爬极强；
2. 无需手动构造 UA、cipher、tls 版本；
3. 原生支持 curl 全部底层参数，兼容性强；
4. 异步接口设计贴近 requests，学习成本极低。

常见踩坑

1. 不要混用同步 Session 和异步 AsyncSession；
2. 同一个 AsyncSession 可无限复用，不要频繁新建；
3. 并发极高时可控制并发量（用 asyncio.Semaphore 限制并发数）：

 

# 限制最大并发2个
sem = asyncio.Semaphore(2)
async def limited_fetch(session, url):
    async with sem:
        return await fetch(session, url)