# ai-qa-sdk

Python SDK for validating AI outputs.

## Installation

```bash
pip install ai-qa-sdk
```

This installs everything you need including support for Anthropic (Claude) and OpenAI (GPT).

## Quick Start

### 1. Deterministic Checks (No API Key Needed)

```python
from ai_qa import validate

# Validate JSON format - works immediately!
result = validate(
    output='{"name": "John", "age": 30}',
    checks=["json_format"]
)

print(f"Passed: {result.passed}")
for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

### 2. LLM-Powered Checks (Configure Your Provider)

**Get an API key first:**
- **Anthropic (Claude):** https://console.anthropic.com → API Keys
- **OpenAI (GPT):** https://platform.openai.com/api-keys

**Then configure and use:**
```python
from ai_qa import configure, validate

# Configure your preferred provider
configure(provider="anthropic", api_key="sk-ant-YOUR-KEY-HERE")

# Or use OpenAI instead
# configure(provider="openai", api_key="sk-YOUR-KEY-HERE")

# Now you can use any LLM-powered check
result = validate(
    output="The capital of France is Paris",
    input="What is the capital of France?",
    checks=["factuality", "relevance", "hallucination"]
)

print(f"Passed: {result.passed}")
print(f"Cost: ${result.total_cost:.4f}")

for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

## Available Checks

### ✓ No API Key Required
- `json_format` - Validate JSON syntax
- `regex` - Pattern matching
- `schema` - JSON schema validation
- `length` - Output length bounds

### 🔑 Requires API Key (Anthropic or OpenAI)
- `hallucination` - Detect made-up facts
- `factuality` - Check factual accuracy
- `relevance` - Relevance to input
- `toxicity` - Harmful content detection
- `bias` - Bias/fairness issues
- `pii` - Personal information detection
- `consistency` - Internal consistency
- `contradiction` - Logical contradictions
- `groundedness` - Grounded in provided context
- And more!

## Providers

ai-qa-sdk supports multiple providers out of the box:

| Provider | Setup | Cost |
|----------|-------|------|
| **Anthropic (Claude)** | `configure(provider="anthropic", api_key="...")` | ~$0.003-0.015 per 1K tokens |
| **OpenAI (GPT)** | `configure(provider="openai", api_key="...")` | ~$0.03-0.06 per 1K tokens |
| **Local (Ollama)** | `configure(provider="local", base_url="http://localhost:11434")` | Free (runs locally) |

**Get API keys:**
- Anthropic: https://console.anthropic.com/api-keys
- OpenAI: https://platform.openai.com/api-keys
