Metadata-Version: 2.4
Name: aat-devqa
Version: 1.1.0
Summary: AWT (AI Watch Tester) — AI-powered E2E testing with self-healing DevQA loop
Project-URL: Homepage, https://ai-watch-tester.vercel.app
Project-URL: Repository, https://github.com/ksgisang/AI-Watch-Tester
Project-URL: Bug Tracker, https://github.com/ksgisang/AI-Watch-Tester/issues
Project-URL: Agent Skill, https://github.com/ksgisang/awt-skill
Author-email: AILoopLab <ksgisang@gmail.com>
License-Expression: AGPL-3.0-only
License-File: LICENSE
Keywords: ai,automation,devqa,e2e,playwright,qa,self-healing,testing,web-testing
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: anthropic<1.0,>=0.40
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: jinja2<4.0,>=3.1
Requires-Dist: numpy<3.0,>=1.24
Requires-Dist: openai<2.0,>=1.0
Requires-Dist: opencv-python-headless<5.0,>=4.8
Requires-Dist: pillow<12.0,>=10.0
Requires-Dist: playwright<2.0,>=1.40
Requires-Dist: pyautogui<1.0,>=0.9.54
Requires-Dist: pydantic-settings<3.0,>=2.1
Requires-Dist: pydantic<3.0,>=2.5
Requires-Dist: pytesseract<1.0,>=0.3.10
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: rich<14.0,>=13.0
Requires-Dist: typer[all]<1.0,>=0.9
Provides-Extra: dev
Requires-Dist: mypy<2.0,>=1.8; extra == 'dev'
Requires-Dist: pandas>=2.0; extra == 'dev'
Requires-Dist: pre-commit<4.0,>=3.6; extra == 'dev'
Requires-Dist: pytest-asyncio<1.0,>=0.23; extra == 'dev'
Requires-Dist: pytest-cov<6.0,>=4.0; extra == 'dev'
Requires-Dist: pytest-mock<4.0,>=3.12; extra == 'dev'
Requires-Dist: pytest<9.0,>=8.0; extra == 'dev'
Requires-Dist: ruff<1.0,>=0.4; extra == 'dev'
Requires-Dist: types-pyyaml<7.0,>=6.0; extra == 'dev'
Provides-Extra: web
Requires-Dist: cryptography>=41.0; extra == 'web'
Requires-Dist: fastapi<1.0,>=0.110; extra == 'web'
Requires-Dist: pypdf<7.0,>=4.0; extra == 'web'
Requires-Dist: python-docx<2.0,>=1.0; extra == 'web'
Requires-Dist: python-multipart>=0.0.9; extra == 'web'
Requires-Dist: uvicorn[standard]<1.0,>=0.27; extra == 'web'
Requires-Dist: websockets<14.0,>=12.0; extra == 'web'
Description-Content-Type: text/markdown

    AWT Logo

  AWT — AI Watch Tester
  
  Enter a URL. AI generates, executes, and heals E2E tests — automatically.

---

## Demo

  AWT Demo
  
  Enter a URL → AI scans your site → generates test scenarios → executes with live screenshots → reports results.

---

## Why AWT?

Most E2E testing tools still require you to **write code** or **record flows** before you can run a single test. AWT flips that model:

1. **You provide a URL** (and optionally a spec document).
2. **AI analyzes** the page structure, forms, navigation, and auth flows.
3. **AI generates** complete YAML test scenarios with selectors, test data, and assertions.
4. **Playwright executes** the scenarios in a real browser with humanized input.
5. **If a test fails**, the DevQA Loop kicks in — AI reads the failure, fixes the scenario or source code, and re-runs.

No test code. No recording. No manual maintenance.

---

## Cloud Version vs Local CLI

AWT ships in two modes. Pick one — or use both.

| | **Cloud** | **Local CLI** |
|---|-----------|---------------|
| **URL** | [ai-watch-tester.vercel.app](https://ai-watch-tester.vercel.app) | `aat dashboard` → localhost:9500 |
| **Install** | None — just sign up | `pip install aat-devqa` |
| **Browser** | Headless Chromium on server | Real Chromium on your machine |
| **AI key** | Server-provided or BYOK | Your own key (OpenAI / Anthropic / Ollama) |
| **Best for** | PMs, planners, quick tests | Developers, CI/CD, offline use |
| **Pricing** | Free (5/mo) · Pro $28.99 · Team $98.99 | Free forever (MIT, unlimited) |
| **Data** | Stored on our servers | Never leaves your machine |

### Cloud — Start in 30 seconds

```
1. Visit https://ai-watch-tester.vercel.app
2. Sign up (email or GitHub)
3. Enter your target URL
4. Watch AI generate and execute tests
```

### Local CLI — Full control

```bash
pip install aat-devqa
playwright install chromium

# Option 1: Web dashboard
aat dashboard                    # http://localhost:9500

# Option 2: CLI
aat start                        # guided mode
aat generate --url https://example.com --provider openai
aat run scenarios/
```

### From Source

```bash
git clone https://github.com/ksgisang/AI-Watch-Tester.git
cd AI-Watch-Tester
python -m venv .venv && source .venv/bin/activate
make dev          # install deps + playwright + pre-commit
make test         # verify everything works
aat dashboard     # launch web UI
```

---

## Features

| | Feature | Description |
|---|---------|-------------|
| :robot: | **AI Scenario Generation** | Upload a URL or spec doc (PDF/DOCX/MD) — AI creates E2E test scenarios |
| :globe_with_meridians: | **Real Browser Testing** | Playwright-driven Chromium with Bezier mouse curves and variable-speed typing |
| :recycle: | **Self-Healing DevQA Loop** | AI analyzes failures, patches code or scenarios, and re-runs automatically |
| :cloud: | **Cloud + Local** | Cloud mode (no install, browser dashboard) or local mode (real browser, full control) |
| :bar_chart: | **Live Dashboard** | Real-time screenshot streaming, step-by-step progress, event log |
| :page_facing_up: | **Document-Based Generation** | Feed PDF/DOCX/Markdown specs — AI generates scenarios from requirements |
| :key: | **BYOK** | Bring your own AI API key (OpenAI, Anthropic, Ollama) — encrypted at rest |
| :test_tube: | **CI/CD Ready** | One-line `curl` integration with any pipeline |
| :jigsaw: | **Plugin Architecture** | Engines, matchers, AI adapters, and reporters are all swappable via registries |

---

## Supported AI Providers

| Provider | Models | Cost | Setup |
|----------|--------|------|-------|
| **OpenAI** | gpt-4o, gpt-4o-mini | Pay-per-use | `export OPENAI_API_KEY=sk-...` |
| **Anthropic** | Claude Sonnet 4 | Pay-per-use | `export ANTHROPIC_API_KEY=sk-ant-...` |
| **Ollama** | codellama, llama3, mistral | Free (local GPU) | `ollama serve` |

Configure in `aat.yaml` or via environment variables:

```yaml
ai:
  provider: openai        # openai | anthropic | ollama
  model: gpt-4o
  api_key: ${OPENAI_API_KEY}
```

Cloud users can bring their own API key (BYOK) via **Settings > AI Provider**.

---

## How It Compares

### vs testRigor

| | **AWT** | **testRigor** |
|---|---------|---------------|
| Test authoring | AI generates from URL/docs — zero input | Plain English commands (you write) |
| Self-healing | DevQA Loop (AI re-generates) | Built-in auto-maintenance |
| Pricing | Free (MIT, self-host) | Enterprise pricing (~$800+/mo) |
| Open source | Yes | No |
| Setup time | Seconds (enter URL) | Minutes (write English scripts) |

**Choose AWT** if you want fully automated test generation with no scripting at all, or need a self-hostable open-source tool. **Choose testRigor** if you prefer writing plain-English test specs with enterprise support.

### vs Applitools

| | **AWT** | **Applitools** |
|---|---------|----------------|
| Primary focus | Functional E2E test generation + execution | Visual regression + cross-browser comparison |
| AI role | Generates entire test scenarios | Compares screenshots for visual differences |
| Standalone | Yes (full pipeline) | No (requires Cypress/Playwright/Selenium) |
| Pricing | Free (MIT) | Free tier + paid plans |

**Choose AWT** for AI-driven functional testing where you need scenarios generated automatically. **Choose Applitools** when pixel-perfect visual consistency across browsers is the priority. They complement each other — AWT generates and runs tests, Applitools can validate visual output.

### vs Playwright / Cypress

These are **excellent** browser automation frameworks that AWT is built on top of. The difference is **who writes the tests**: you (Playwright/Cypress) or AI (AWT). If your team wants full programmatic control, use them directly. If you want AI to handle test creation and maintenance, AWT fills that gap.

See [docs/COMPARISON.md](docs/COMPARISON.md) for a detailed breakdown against Playwright, Cypress, Testim, Katalon, and Mabl.

---

## Architecture

```
aat start / aat dashboard
       │
       ▼
┌─────────────────────────────────────────────┐
│                   CLI (Typer)                │
├─────────────────────────────────────────────┤
│              Core Orchestrator              │
│  ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│  │ Executor │ │Comparator│ │ DevQA Loop  │ │
│  └────┬─────┘ └────┬─────┘ └──────┬──────┘ │
├───────┼─────────────┼──────────────┼────────┤
│  ┌────▼────┐  ┌─────▼─────┐  ┌────▼─────┐  │
│  │ Engine  │  │  Matcher  │  │ Adapter  │  │
│  │Registry │  │ Registry  │  │ Registry │  │
│  └─────────┘  └───────────┘  └──────────┘  │
│  web|desktop  template|ocr   openai|claude  │
│               feature|hybrid ollama         │
├─────────────────────────────────────────────┤
│  Models (Pydantic v2)  │  Config (Settings) │
└─────────────────────────────────────────────┘
```

All modules follow **ABC + plugin registry** pattern — extend the base class, register in `__init__.py`, done.

---

## Development

### Prerequisites

- Python 3.11+
- [Tesseract OCR](https://github.com/tesseract-ocr/tesseract) — `brew install tesseract` / `apt install tesseract-ocr`
- Git

### Make Commands

| Command | Description |
|---------|-------------|
| `make dev` | Install all deps + Playwright + pre-commit |
| `make lint` | ruff check |
| `make format` | ruff format + auto-fix |
| `make typecheck` | mypy strict |
| `make test` | pytest |
| `make test-cov` | pytest + coverage report |
| `make clean` | Remove caches and build artifacts |

---

## Contributing

Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for:

- Development environment setup
- Code style (ruff + mypy strict)
- Test writing guidelines
- Pull request process
- Plugin development (adding new engines, matchers, or AI adapters)

```bash
git checkout -b feat/my-feature
# make changes
make format && make lint && make typecheck && make test
git commit -m "feat(scope): description"
# open PR
```

---

## Documentation

| Document | Description |
|----------|-------------|
| [Quick Start Guide](docs/QUICK_START.md) | Install, configure, run your first test |
| [API Reference](docs/API_REFERENCE.md) | REST API + WebSocket documentation |
| [Comparison](docs/COMPARISON.md) | AWT vs Playwright, Cypress, Testim, Katalon, Mabl |
| [FAQ](docs/FAQ.md) | Common questions |
| [CI/CD Guide](cloud/docs/CI_CD_GUIDE.md) | Pipeline integration (GitHub Actions, GitLab CI) |
| [Cloud Backend](cloud/README.md) | Self-hosting the cloud backend |

---

## FAQ

What is AWT?

AWT (AI Watch Tester) is an open-source, AI-powered E2E testing tool. You give it a URL, and it automatically generates test scenarios, executes them in a real browser (Playwright), and reports results — no test code required.

How do I install it?

**Cloud (no install):** Visit [ai-watch-tester.vercel.app](https://ai-watch-tester.vercel.app) and enter a URL.

**Local:**
```bash
pip install aat-devqa
playwright install chromium
aat dashboard
```

**From source:**
```bash
git clone https://github.com/ksgisang/AI-Watch-Tester.git
cd AI-Watch-Tester
make dev && aat dashboard
```


Which AI providers are supported?

| Provider | Models | Cost |
|----------|--------|------|
| **OpenAI** | gpt-4o, gpt-4o-mini | Pay-per-use |
| **Anthropic** | Claude Sonnet 4 | Pay-per-use |
| **Ollama** | codellama, llama3, mistral | Free (local GPU) |

Cloud users can bring their own API key (BYOK) via the Settings page. Keys are Fernet-encrypted at rest.

How much does it cost?

| Plan | Price | Tests/month | Concurrent |
|------|-------|-------------|------------|
| **Free** | $0 | 5 | 1 |
| **Pro** | $28.99/mo | 100 | 3 |
| **Team** | $98.99/mo | 500 | 10 |

The open-source local mode is completely free with no limits — you just need your own AI API key.

Is it open source?

Yes. AWT is licensed under the MIT License — free for personal and commercial use. You can self-host, modify, and distribute it. Contributions are welcome!

Can I use it in CI/CD?

Yes. Pro and Team plans include API keys for CI/CD integration:

```bash
curl -X POST https://your-awt-server.com/api/v1/run \
  -H "X-API-Key: awt_your_key" \
  -H "Content-Type: application/json" \
  -d '{"target_url": "https://staging.example.com"}'
```

See the [CI/CD Guide](cloud/docs/CI_CD_GUIDE.md) for GitHub Actions and GitLab CI examples.

Is my data secure?

- All traffic is encrypted via HTTPS/TLS
- BYOK API keys are Fernet-encrypted (AES-128-CBC + HMAC-SHA256) at rest
- Screenshots are auto-deleted after 7 days
- Database hosted on Supabase (AWS Seoul region)
- See our Privacy Policy (https://ai-watch-tester.vercel.app/privacy) for full details

---

## License

[MIT](LICENSE) — free for personal and commercial use.

---

  Built with Playwright, OpenCV, and a lot of AI. Made by @ksgisang (https://github.com/ksgisang).

