Metadata-Version: 2.4
Name: ai-trace-auditor
Version: 0.16.0
Summary: AI compliance auditor: EU AI Act, NIST AI RMF, ISO 42001, SOC 2. Every requirement verified against primary legal text with exact clause citations.
Project-URL: Homepage, https://github.com/BipinRimal314/ai-trace-auditor
Project-URL: Repository, https://github.com/BipinRimal314/ai-trace-auditor
Project-URL: Issues, https://github.com/BipinRimal314/ai-trace-auditor/issues
Author-email: Bipin Rimal <bipinrimal314@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,annex-iv,audit,compliance,data-flow,eu-ai-act,gdpr,llm,mermaid,nist,observability,traceability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Requires-Dist: jinja2>=3.1
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: typer>=0.12
Provides-Extra: arize
Requires-Dist: httpx>=0.27; extra == 'arize'
Provides-Extra: importers
Requires-Dist: httpx>=0.27; extra == 'importers'
Provides-Extra: langfuse
Requires-Dist: httpx>=0.27; extra == 'langfuse'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: notify
Requires-Dist: httpx>=0.27; extra == 'notify'
Provides-Extra: pdf
Requires-Dist: markdown>=3.5; extra == 'pdf'
Requires-Dist: weasyprint>=62.0; extra == 'pdf'
Description-Content-Type: text/markdown

# AI Trace Auditor

**The EU AI Act takes effect August 2, 2026.** Your AI system needs compliance evidence. Your observability tools collect traces. Your GRC platform manages policies. Nothing translates traces into compliance evidence. This tool does.

Open-source CLI that audits LLM traces against EU AI Act Articles 11, 12, 13, and 25, plus NIST AI RMF and GDPR Article 30. 301 tests. Zero LLM dependencies. Runs locally.

## The Problem

Companies have traces from Langfuse, Arize, or OpenTelemetry, but no automated way to answer: "Do our traces satisfy the regulatory requirements?" 67% of AI teams discover quality regressions from user complaints despite having tracing infrastructure. The gap isn't data collection; it's interpretation.

Consultants charge $30K-$500K per compliance engagement. CEPS estimates EUR 29K per AI model for Annex IV documentation alone. This tool automates the gap analysis for free.

## What It Covers

| Regulation | Scope |
|---|---|
| EU AI Act Article 11 | Technical documentation (Annex IV). Auto-generates 60-70% of required sections from code scanning |
| EU AI Act Article 12 | Record-keeping. Audits trace data against 15+ discrete requirements |
| EU AI Act Article 13 | Transparency (provider to deployer). Data flow mapping with GDPR role classification |
| EU AI Act Article 25 | Value chain accountability. Multi-agent DAG auditing with per-agent penalty propagation |
| NIST AI RMF | GOVERN, MAP, MEASURE, MANAGE subcategories (~15 requirements) |
| GDPR Article 30 | Records of Processing Activities. Auto-generated from detected data flows |

## Multi-Agent Compliance (v0.14.0)

Automatically audits multi-agent systems (LangGraph, CrewAI, AutoGen, Google ADK):
- Reconstructs execution DAGs from parent-child span relationships
- Per-agent compliance scores with bottom-up penalty propagation
- Article 25 "value chain accountability" checks
- Liability shift detection (deployer becoming provider)
- Mermaid DAG visualizations

No other open-source tool does multi-agent compliance auditing.

## Supported Trace Formats

| Format | Source |
|---|---|
| OTel OTLP JSON | OpenTelemetry GenAI semantic conventions |
| Langfuse JSON | Langfuse trace exports |
| Claude Code | `~/.claude/projects/` conversation traces |
| Raw JSONL | Any provider's API logs |

Auto-detected. Use `--format` to override.

## Install

```bash
pip install ai-trace-auditor
```

## Quick Start

```bash
# Audit traces against EU AI Act
aitrace audit traces.json -r "EU AI Act" -o report.md

# Full compliance package: Articles 11 + 12 + 13 + GDPR in one run
aitrace comply ./my-ai-project/ --traces traces.json

# Generate Annex IV technical documentation from code
aitrace docs ./my-ai-project/

# Map data flows for Article 13 + GDPR Article 30
aitrace flow ./my-ai-project/

# Multi-agent audit with DAG visualization
aitrace audit multi_agent_traces.json --show-dag
```

## Example Output

Real output from auditing 1,522 Claude Code spans:

```
Overall Compliance Score: 79.3%

| Status    | Count |
|-----------|-------|
| Satisfied |    10 |
| Partial   |     5 |
| Missing   |     3 |

Top gaps:
  1. Not logging: Temperature parameter controlling output randomness
  2. Not logging: Maximum token limit for output generation
  3. Incomplete: Output responses generated by the AI model (31.9% coverage)
  4. Incomplete: Input prompts/messages (4.7% coverage)
  5. Not logging: Operation latency in milliseconds
```

## GitHub Action

```yaml
- name: Audit AI traces
  uses: BipinRimal314/ai-trace-auditor@v0.14.0
  with:
    path: traces/exported.json
    regulation: "EU AI Act"
    output: compliance-report.md
    fail-on-gaps: "true"
```

Exit code 0 = all requirements satisfied, 1 = gaps found. CI-friendly.

## Library API

```python
from ai_trace_auditor.ingest import ingest_file
from ai_trace_auditor.analysis.engine import ComplianceAnalyzer
from ai_trace_auditor.regulations.registry import RequirementRegistry

traces = ingest_file(Path("traces.json"))
registry = RequirementRegistry()
registry.load()

report = ComplianceAnalyzer(registry).analyze(
    traces=traces,
    regulations=["EU AI Act"],
)

print(f"Score: {report.overall_score:.1%}")
```

## Architecture

```
ai-trace-auditor/
├── src/ai_trace_auditor/
│   ├── cli.py              # 7 commands: audit, docs, flow, comply, ingest, requirements, health
│   ├── ingest/             # Trace ingestion (OTel, Langfuse, Claude Code, raw JSONL)
│   ├── analysis/           # Gap analysis engine + multi-agent DAG auditing
│   ├── models/             # Pydantic v2 data models
│   ├── regulations/        # YAML requirement definitions (extensible)
│   ├── docs/               # Article 11 Annex IV generator
│   ├── flow/               # Article 13 data flow mapper + GDPR RoPA
│   ├── comply/             # Full compliance package runner
│   ├── evidence/           # Auditor-ready evidence pack generator
│   ├── reports/            # Markdown, JSON, PDF report generation
│   ├── scanner/            # Code scanner (AI framework detection)
│   └── guide_linter/       # Lints compliance guides for accuracy
├── requirements/           # YAML regulatory requirement definitions
│   ├── eu_ai_act/          # Articles 12, 19
│   └── nist_ai_rmf/       # GOVERN, MAP, MEASURE, MANAGE
└── tests/                  # 301 tests
```

No dependency on any LLM framework. Intentionally framework-agnostic.

## Competitive Landscape

| Tool | What It Does | How We Differ |
|---|---|---|
| Langfuse / Arize | Collect and visualize traces | We interpret traces against regulations |
| Credo AI ($41M raised) | Enterprise AI governance platform | Free, open-source, runs locally |
| OneTrust / Vanta | GRC policy management | We ingest AI-specific traces, not generic policy docs |
| Holistic AI | AI risk management (consulting) | CLI + CI integration, no sales call required |

The competition is consultants, not software.

## Disclaimer

This tool provides automated compliance analysis. It is not legal advice. Risk classification under Annex III requires legal review. Consult qualified counsel for compliance decisions.

## License

Apache 2.0
