Metadata-Version: 2.4
Name: chuscraper
Version: 0.16.4
Summary: A blazing fast, async-first, undetectable webscraping/web automation framework
Project-URL: Homepage, https://github.com/ToufiqQureshi/chuscraper
Project-URL: Repository, https://github.com/ToufiqQureshi/chuscraper
Project-URL: Issues, https://github.com/ToufiqQureshi/chuscraper/issues
Author-email: Toufiq Qureshi <toufiq@example.com>
License: MIT License
        
        Copyright (c) 2024 Toufiq Qureshi
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
Requires-Python: >=3.10
Requires-Dist: asyncio-atexit>=1.0.1
Requires-Dist: deprecated>=1.2.14
Requires-Dist: emoji>=2.14.1
Requires-Dist: grapheme>=0.6.0
Requires-Dist: mss>=9.0.2
Requires-Dist: websockets>=14.0
Provides-Extra: ai
Requires-Dist: beautifulsoup4>=4.12.0; extra == 'ai'
Requires-Dist: google-genai>=1.0.0; extra == 'ai'
Requires-Dist: google-generativeai>=0.8.0; extra == 'ai'
Requires-Dist: openai>=1.0.0; extra == 'ai'
Requires-Dist: pydantic>=2.0.0; extra == 'ai'
Description-Content-Type: text/markdown

  Chuscraper Logo

🕷️ Chuscraper

  LLM + CDP powered stealth-focused web scraping & automation framework
  You Only Scrape Once — data extraction made smarter, faster, and more resilient.

---

## 🚀 What is Chuscraper?
Chuscraper is a Python web scraping & automation library that uses **CDP (Chrome DevTools Protocol)** and **LLMs** to extract structured data, interact with pages, and automate workflows — with a heavy focus on **Anti-Detection** and **Stealth**.

With AI-powered extraction, you tell it *what* to extract — it figures out *how*.

---

## 🌟 Features

### 🕵️‍♂️ Stealth & Anti-Detection
- Hides `navigator.webdriver`, user agent rotation
- Canvas/WebGL noise + hardware spoofing
- Timezone & geolocation spoofing

### 🤖 AI-Driven Data Extraction
- **Semantic extraction** using LLMs
- Converts HTML into structured JSON/Pydantic

### 🧠 Autonomous Navigation
- Intelligent pilot (`ai_pilot`) that clicks/types until goal achieved

### ⚡ Async + Fast
Built on async CDP, low overhead, no heavy browser bundles.

### 🔄 Flexible Outputs
Supports JSON, CSV, Markdown, Excel, Pydantic, and more.

### 🌐 Integrations
- LLM Providers: OpenAI, Gemini, Anthropic, Ollama
- Frameworks: LangChain, LlamaIndex, Agno, Crew.ai

---

## 📦 Installation

```bash
pip install chuscraper

# For AI Capabilities
pip install chuscraper[ai]
```

> [!TIP]
> Use within a virtual environment to avoid conflicts.

---

## 💻 Quick Start (Async)

```python
import asyncio
from chuscraper import start

async def main():
    browser = await start(headless=False)
    page = await browser.get("https://www.makemytrip.com/")
    # ... see full examples in examples/ directory
```

> **🔥 New:** Check out our [Real-World Scraper Examples](docs/REAL_WORLD_EXAMPLES.md) for Amazon, Flipkart, Walmart, Google & more!
    page = await browser.get("https://www.makemytrip.com/")

    # Tell the AI what to extract
    print("AI is navigating...")
    await page.ai_pilot("Search hotels in Goa for next weekend")

    # Extract structured data
    result = await page.ai_extract("Get the first 3 hotels with prices")
    import json
    print(json.dumps(result, indent=2))

    await browser.stop()

if __name__ == "__main__":
    asyncio.run(main())
```

---

## 🤖 AI Usage with Providers
Chuscraper supports multiple providers out-of-the-box.

### 1. Gemini (Native)
```python
from chuscraper.ai.providers import GeminiProvider
provider = GeminiProvider(api_key="YOUR_GEMINI_API_KEY")
await page.ai_extract("Extract data", provider=provider)
```

### 2. OpenAI
```python
from chuscraper.ai.providers import OpenAIProvider
provider = OpenAIProvider(api_key="YOUR_OPENAI_API_KEY")
await page.ai_extract("Extract data", provider=provider)
```


## 🛡️ Stealth & Anti-Detection Proof

We don't just claim to be stealthy; we prove it. Below are the results from top anti-bot detection suites, all passed with **100% "Human" status**.

👉 **[View Full Visual Proofs & Screenshots Here](docs/STEALTH_PROOF.md)**

| Detection Suite | Result | Status |
|----------------|--------|--------|
| **SannySoft** | No WebDriver detected | ✅ Pass |
| **BrowserScan** | 100% Trust Score | ✅ Pass |
| **PixelScan** | Consistent Fingerprint | ✅ Pass |
| **IPHey** | Software Clean (Green) | ✅ Pass |
| **CreepJS** | 0% Stealth / 0% Headless | ✅ Pass |
| **Fingerprint.com** | No Bot Detected | ✅ Pass |

### 🌍 Real-World Protection Bypass
We tested `chuscraper` against live websites protected by major security providers:

| Provider | Target | Result |
|----------|--------|--------|
| **Cloudflare** | Turnstile Demo | ✅ Solved Automatically |
| **DataDome** | Antoine Vastel Research | ✅ Accessed |
| **Akamai** | Nike Product Page | ✅ Bypassed |

---

## 📖 Documentation
Full technical guides are available in the `docs/` folder:

- [English (Main)](README.md)
- [Project API Guide](docs/api_guide_v2.md)
- [Stealth Comparison](docs/stealth_comparison.md)

*Translations (Chinese, Japanese, etc.) coming soon.*

---

## 🛠️ Contributing
Want to contribute? Open an issue or send a pull request — all levels welcome! Please follow the `CONTRIBUTING.md` guidelines.

---

## 📜 License
Chuscraper is licensed under the MIT License.

Made with ❤️ by [Toufiq Qureshi]
