Metadata-Version: 2.4
Name: curl_cffi
Version: 0.15.1b1
Summary: libcurl ffi bindings for Python, with impersonation support.
Author-email: lexiforest <infinitesheldon@gmail.com>
License-Expression: MIT
Project-URL: repository, https://github.com/lexiforest/curl_cffi
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Free Threading
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cffi>=2.0.0
Requires-Dist: certifi>=2024.2.2
Provides-Extra: extra
Requires-Dist: readability-lxml>=0.8.1; extra == "extra"
Requires-Dist: markdownify>=1.1.0; extra == "extra"
Requires-Dist: lxml_html_clean; extra == "extra"
Provides-Extra: cli
Requires-Dist: rich; extra == "cli"
Provides-Extra: dev
Requires-Dist: charset_normalizer<4.0,>=3.3.2; extra == "dev"
Requires-Dist: coverage<7.0,>=6.4.1; extra == "dev"
Requires-Dist: cryptography<47.0,>=46.0.4; extra == "dev"
Requires-Dist: httpx==0.23.1; extra == "dev"
Requires-Dist: mypy<2.0,>=1.9.0; extra == "dev"
Requires-Dist: pytest<9.0,>=8.1.1; extra == "dev"
Requires-Dist: pytest-asyncio<1.0,>=0.23.6; extra == "dev"
Requires-Dist: pytest-trio<1.0,>=0.8.0; extra == "dev"
Requires-Dist: ruff<1.0,>=0.3.5; extra == "dev"
Requires-Dist: trio<1.0,>=0.25.0; extra == "dev"
Requires-Dist: trustme<2.0,>=1.1.0; extra == "dev"
Requires-Dist: uvicorn<1.0,>=0.29.0; extra == "dev"
Requires-Dist: websockets>=14.0; extra == "dev"
Requires-Dist: typing_extensions; extra == "dev"
Provides-Extra: build
Requires-Dist: cibuildwheel; extra == "build"
Requires-Dist: wheel; extra == "build"
Provides-Extra: test
Requires-Dist: charset_normalizer<4.0,>=3.3.2; extra == "test"
Requires-Dist: cryptography<47.0,>=46.0.4; extra == "test"
Requires-Dist: litestar<3.0,>=2.19.0; extra == "test"
Requires-Dist: httpx==0.23.1; extra == "test"
Requires-Dist: proxy.py<3.0,>=2.4.3; extra == "test"
Requires-Dist: pytest<9.0,>=8.1.1; extra == "test"
Requires-Dist: pytest-asyncio<1.0,>=0.23.6; extra == "test"
Requires-Dist: pytest-trio<1.0,>=0.8.0; extra == "test"
Requires-Dist: python-multipart<1.0,>=0.0.9; extra == "test"
Requires-Dist: trio<1.0,>=0.25.0; extra == "test"
Requires-Dist: trustme<2.0,>=1.1.0; extra == "test"
Requires-Dist: uvicorn<1.0,>=0.29.0; extra == "test"
Requires-Dist: websockets>=14.0; extra == "test"
Requires-Dist: typing_extensions; extra == "test"
Dynamic: license-file

# curl_cffi

[![PyPI Downloads](https://static.pepy.tech/badge/curl-cffi/week)](https://pepy.tech/projects/curl-cffi)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/curl_cffi)
[![PyPI version](https://badge.fury.io/py/curl-cffi.svg)](https://badge.fury.io/py/curl-cffi)
[![Generic badge](https://img.shields.io/badge/Telegram%20Channel-join-blue?logo=telegram)](https://t.me/impersonate_pro)
[![Generic badge](https://img.shields.io/badge/Discord-join-purple?logo=blue)](https://discord.gg/kJqMHHgdn2)

[Documentation](https://curl-cffi.readthedocs.io)

Python binding for [curl-impersonate fork](https://github.com/lexiforest/curl-impersonate)
via [cffi](https://cffi.readthedocs.io/en/latest/). For commercial support, visit [impersonate.pro](https://impersonate.pro).

`curl_cffi` is the most popular Python binding for `curl`. Unlike other pure
python http clients like `httpx` or `requests`, `curl_cffi` can impersonate
browsers' TLS/JA3 and HTTP/2 fingerprints. If you are blocked by some
website for no obvious reason, you can give `curl_cffi` a try.

Python 3.10 is the minimum supported version since v0.14.

## Recent highlights

- 💨 http/3 fingerprints and UDP socks5 proxy support was added in `v0.15.0`!
- 🦞 Added `curl-cffi` CLI and skills for debugging and for claws/agents.

## Recall.ai - API for meeting recordings

If you’re looking for a meeting recording API, consider checking out [Recall.ai](https://www.recall.ai/?utm_source=github&utm_medium=sponsorship&utm_campaign=lexiforest-curl_cffi), an API that records Zoom, Google Meet, Microsoft Teams, in-person meetings, and more.

## Residential Proxies

Thordata: A reliable and cost-effective proxy service provider. One-click collection of public network data, providing enterprises and developers with stable, efficient, and compliant global proxy IP services. Register for a free trial of [residential proxies](https://www.thordata.com/?ls=github&lk=curl_) and receive 2000 free SERP API calls.

## Sponsors

Maintenance of this project is made possible by all the contributors (https://github.com/lexiforest/curl_cffi/graphs/contributors) and sponsors (https://github.com/sponsors/lexiforest). If you'd like to sponsor this project and have your avatar or company logo appear below click here (https://github.com/sponsors/lexiforest). 💖

------

### Bypass Cloudflare with API

Yescaptcha is a proxy service that bypasses Cloudflare and uses the API interface to
obtain verified cookies (e.g. `cf_clearance`). Click [here](https://yescaptcha.com/i/stfnIO)
to register: <https://yescaptcha.com/i/stfnIO>

------

TLS fingerprinting alone isn't enough for modern bot protection. [Hyper Solutions](https://hypersolutions.co?utm_source=github&utm_medium=readme&utm_campaign=curl_cffi) provides the missing piece - API endpoints that generate valid antibot tokens for:

Akamai • DataDome • Kasada • Incapsula

No browser automation. Just simple API calls that return the exact cookies and headers these systems require.

🚀 [Get Your API Key](https://hypersolutions.co?utm_source=github&utm_medium=readme&utm_campaign=curl_cffi) | 📖 [Docs](https://docs.justhyped.dev) | 💬 [Discord](https://discord.gg/akamai)

------

## Impersonate Suite

`curl-cffi` is part of the impersonate suite.

- [curl-impersonate](https://github.com/lexiforest/curl-impersonate). A curl distribution that impersonates browsers.
- [curl_cffi](https://github.com/lexiforest/curl_cffi). Python binding to curl-impersonate.
- [impers](https://github.com/lexiforest/impers). Node.js binding to curl-impersonate.
- [impersonate.pro](https://impersonate.pro). Commercial support, more fingerprints and integrated solutions.

## Features

- Supports JA3/TLS and http2 fingerprints impersonation, including recent browsers and custom fingerprints.
- Much faster than requests/httpx, on par with aiohttp/pycurl, see [benchmarks](https://github.com/lexiforest/curl_cffi/tree/main/benchmark).
- Mimics the requests API, no need to learn another one.
- Pre-compiled, so you don't have to compile on your machine.
- Supports `asyncio` with proxy rotation on each request.
- Supports http 2.0, which requests does not.
- Supports http 3.0, with fingerprints and udp proxy.
- Supports websocket.
- MIT licensed.

||requests|aiohttp|httpx|pycurl|curl_cffi|
|---|---|---|---|---|---|
|http/2|❌|❌|✅|✅|✅|
|http/3|❌|❌|❌|☑️1|✅2|
|sync|✅|❌|✅|✅|✅|
|async|❌|✅|✅|❌|✅|
|websocket|❌|✅|❌|❌|✅|
|native retry|❌|❌|❌|❌|✅|
|fingerprints|❌|❌|❌|❌|✅|
|speed|🐇|🐇🐇|🐇|🐇🐇|🐇🐇|

Notes:

1. For pycurl, http/3 is usually disabled at compile time by default.
2. http/3 support since v0.11.4, http/3 proxy and fingerprints since v0.15.0.

### curl-cffi CLI(new)

Since v0.15, `curl_cffi` comes with a CLI called `curl-cffi`, you can use it for debugging
a certain url with the `--impersonate` option. It can also serve as a `web_fetch`
replacement for "claws" and "agents".

||curl|httpie|curl-cffi|
|---|---|---|---|
|http/2|✅|❌|✅|
|http/3|☑️1|❌|✅|
|human-friendly|☑️2|✅|✅|
|colorful|❌|✅|✅3|
|fingerprints|❌|❌|✅|

Notes:

1. You need an http/3 enabled curl build, it's not enabled by default, at leat on my machine.
2. As a long time command line user, I personally feel very comfortable using `curl -X POST httpbin.org`, but some users may prefer `http GET httpbin.org` syntax. If you prefer the curl syntax, you can keep using `curl-impersonate`.
3. Install `curl_cffi[cli]` for colorful CLI output. Without `rich`, the CLI uses plain text output.

## Install

    pip install curl_cffi --upgrade

This should work on Linux, macOS and Windows out of the box.

On macOS, you can also install via Homebrew:

    brew install lexiforest/tap/curl-cffi

Android support, including Termux, is currently in beta, you can install the beta release for testing.
For BSD systems, we need to get libcurl-impersonate compile first, and then add support in curl_cffi.
If you are using these OSes, please lend an hand.

To install beta releases:

    pip install curl_cffi --upgrade --pre

To install unstable version from GitHub:

    git clone https://github.com/lexiforest/curl_cffi/
    cd curl_cffi
    make preprocess
    pip install .

## Usage

`curl_cffi` comes with a low-level `curl` API and a high-level `requests`-like API.
`curl_cffi` also bundles with a CLI called `curl-cffi`.

### CLI

```sh
curl-cffi get tls.browserleaks.com/json

# curl-cffi can be hard to type, use an alias if you want
alias imp=curl-cffi
imp get tls.browserleaks.com/json --impersonate chrome
```

For a complete CLI guide, see [docs](https://curl-cffi.readthedocs.io).

### requests-like

```python
import curl_cffi

# Notice the impersonate parameter
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome")

print(r.json())
# output: {..., "ja3n_hash": "aa56c057ad164ec4fdcb7a5a283be9fc", ...}
# the js3n fingerprint should be the same as target browser

# To keep using the latest browser version as `curl_cffi` updates,
# simply set impersonate="chrome" without specifying a version.
# Other similar values are: "safari" and "safari_ios"
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome")

# Use http/3 with impersonation
r = curl_cffi.get(
    "https://fp.impersonate.pro/api/http3",
    http_version="v3",
    impersonate="chrome"
)

# To pin a specific version, use version numbers together.
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome124")

# To impersonate other than browsers, bring your own ja3/akamai strings
# See examples directory for details.
r = curl_cffi.get("https://tls.browserleaks.com/json", ja3=..., akamai=...)

# http/socks proxies are supported
proxies = {"https": "http://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome", proxies=proxies)

proxies = {"https": "socks://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome", proxies=proxies)
```

### Sessions

```python
s = curl_cffi.Session()

# httpbin is a http test website, this endpoint makes the server set cookies
s.get("https://httpbin.org/cookies/set/foo/bar")
print(s.cookies)
# <Cookies[<Cookie foo=bar for httpbin.org />]>

# retrieve cookies again to verify
r = s.get("https://httpbin.org/cookies")
print(r.json())
# {'cookies': {'foo': 'bar'}}
```

### Supported impersonate browsers

`curl_cffi` supports the same browser versions as supported by my [fork](https://github.com/lexiforest/curl-impersonate) of [curl-impersonate](https://github.com/lwthiker/curl-impersonate):

The open source version of curl_cffi includes versions whose fingerprints differ from previous versions.
If you see a version, e.g. `chrome135`, was skipped, you can simply impersonate it with your own headers and the previous version.

If you don't want to look up the headers/etc by yourself, consider buying commercial support from [impersonate.pro](https://impersonate.pro).
We have comprehensive browser fingerprints database for almost all the browser versions on various platforms.

If you are trying to impersonate a target other than a browser, use `ja3=...` and `akamai=...`
to specify your own customized fingerprints. See the [docs on impersonation](https://curl-cffi.readthedocs.io/en/latest/impersonate/_index.html) for details.

To see the complete list of fingerprints, use the command line:

```sh
curl-cffi list
```

Since v0.15.1, you can use `curl-cffi update` to retrieve the latest fingerprints, without updating to a new version.
We offer the Safari, Chrome, Firefox updates for free and others as part of the [commercial plan](https://impersonate.pro).

The following table is the builtin fingerprints bundled with current version.

|Browser|Open Source| Pro version|
|---|---|---|
|Chrome|chrome99, chrome100, chrome101, chrome104, chrome107, chrome110, chrome116[1], chrome119[1], chrome120[1], chrome123[3], chrome124[3], chrome131[5], chrome133a[5][6], chrome136[8], chrome142[11], chrome145[13][14], chrome146[13][14]|chrome132, chrome134, chrome135|
|Chrome Android| chrome99_android, chrome131_android[5]|chrome132_android, chrome133_android, chrome134_android, chrome135_android|
|Chrome iOS|N/A|coming soon|
|Safari[9]|safari153[2], safari155[2], safari170[1], safari180[4], safari184[8], safari260[10], safari2601[11]|coming soon|
|Safari iOS[9]| safari172_ios[1], safari180_ios[4], safari184_ios[8], safari260_ios[10]|coming soon|
|Firefox|firefox133[5], firefox135[7], firefox144[11][12], firefox147[13][14]|coming soon|
|Firefox Android|N/A|firefox135_android|
|Tor|tor145[8]|coming soon|
|Edge|edge99, edge101|edge133, edge135|
|Opera|N/A|coming soon|
|Brave|N/A|coming soon|

Notes:

1. Added in version `0.6.0`.
2. Fixed in version `0.6.0`, previous http2 fingerprints were [not correct](https://github.com/lwthiker/curl-impersonate/issues/215).
3. Added in version `0.7.0`.
4. Added in version `0.8.0`.
5. Added in version `0.9.0`.
6. The version suffix `a`(e.g. `chrome133a`) means that this is an alternative version, i.e. the fingerprint has not been officially updated by browser, but has been observed because of A/B testing.
7. Added in version `0.10.0`.
8. Added in version `0.11.0`.
9. Since `0.11.0`, the format `safari184_ios` is preferred over `safari18_4_ios`, both are supported, but the latter is quite confusing and hard to parse.
10. Added in version `0.12.0`.
11. Added in version `0.14.0`.
12. Fixed in version `0.15.0`, previous User-Agent header was [not correct](https://github.com/lexiforest/curl-impersonate/issues/234).
13. Added in version `0.15.0`.
14. http3 support included.

### Asyncio

```python
from curl_cffi import AsyncSession

async with AsyncSession() as s:
    r = await s.get("https://example.com")
```

More concurrency:

```python
import asyncio
from curl_cffi import AsyncSession

urls = [
    "https://google.com/",
    "https://facebook.com/",
    "https://twitter.com/",
]

async with AsyncSession() as s:
    tasks = []
    for url in urls:
        task = s.get(url)
        tasks.append(task)
    results = await asyncio.gather(*tasks)
```

For low-level APIs, Scrapy integration and other advanced topics, see the
[docs](https://curl-cffi.readthedocs.io) for more details.

### WebSockets

```python
from curl_cffi import WebSocket

def on_message(ws: WebSocket, message: str | bytes):
    print(message)

ws = WebSocket(on_message=on_message)
ws.run_forever("wss://api.gemini.com/v1/marketdata/BTCUSD")
```

### Asyncio WebSockets

```python
import asyncio
from curl_cffi import AsyncSession

async with AsyncSession() as session:
    async with session.ws_connect("wss://echo.websocket.org") as ws:
        await asyncio.gather(*[ws.send_str("Hello, World!") for _ in range(10)])
        async for message in ws:
            print(message)
```

See the WebSocket [docs](https://curl-cffi.readthedocs.io/en/latest/websockets.html) for full details and advanced options.

## Ecosystem

- Integrating with Scrapy: [divtiply/scrapy-curl-cffi](https://github.com/divtiply/scrapy-curl-cffi), [jxlil/scrapy-impersonate](https://github.com/jxlil/scrapy-impersonate) and [tieyongjie/scrapy-fingerprint](https://github.com/tieyongjie/scrapy-fingerprint).
- Integrating with [requests](https://github.com/el1s7/curl-adapter), [httpx](https://github.com/vgavro/httpx-curl-cffi) as adapter.
- Integrating with captcha resolvers: [YesCaptcha](https://yescaptcha.com/i/stfnIO).

## Acknowledgement

- Originally forked from [multippt/python_curl_cffi](https://github.com/multippt/python_curl_cffi), which is under the MIT license.
- Headers/Cookies files are copied from [httpx](https://github.com/encode/httpx/blob/master/httpx/_models.py), which is under the BSD license.
- Asyncio support is inspired by Tornado's curl http client.
- The synchronous WebSocket API is inspired by [websocket_client](https://github.com/websocket-client/websocket-client).
- The asynchronous WebSocket API is inspired by [aiohttp](https://github.com/aio-libs/aiohttp).

## Contributing

When submitting an PR, please use a different branch other than `main` and check the
"Allow edits by maintainers" box, so I can update your PR with lint or style fixes. Thanks!

### AI Policy

- Using AI is neither encouraged nor discouraged, use it by your own choice.
- The bottom line here is that every line of code should be **reviewed by human**, and should be [proven to work](https://simonwillison.net/2025/Dec/18/code-proven-to-work/).
- It's not guaranteed that AI will come up with the cleanest solution, you are responsible to guide it to the right way you know.
- Fix any lint errors, make sure your code follows the established convention in this project.
- LLM tends to generate extensive or none comments, revise the comments and make sure they are concise and helpful.
- It's absolutely **not acceptable** to generate the entire PR summary by LLM. To communicate with other human, use words from a human.
- The only acceptable exception is to fix grammar issues if you are not a native English speaker.
- The essence here is to keep [Human in the loop](https://discourse.llvm.org/t/rfc-llvm-ai-tool-policy-human-in-the-loop/89159)

You can even feed the policy above to your "copilot" to let it adjust the style for you. :P
