Metadata-Version: 2.4
Name: adqa
Version: 0.1.0b0
Summary: Autonomous Data Quality Agent: profiling, detection, explanation, and fix proposals for data quality issues.
License: MIT
License-File: LICENSE
Keywords: data-quality,data-engineering,ai-agent,llm,mlops,dataops,observability,data-lineage,automated-remediation
Author: Mohammad T. Ismael
Author-email: mohamed.talaat.ismael@gmail.com
Requires-Python: >=3.12,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Provides-Extra: airbyte
Provides-Extra: all
Provides-Extra: aws
Provides-Extra: core
Provides-Extra: databases
Provides-Extra: excel
Provides-Extra: llm
Provides-Extra: ml
Provides-Extra: mongo
Provides-Extra: remote
Requires-Dist: airbyte (>=0.2.42) ; (python_version < "3.13") and (extra == "airbyte" or extra == "all")
Requires-Dist: boto3 (>=1.34,<2.0) ; extra == "aws" or extra == "all"
Requires-Dist: duckdb (>=1.1.0) ; extra == "databases" or extra == "all"
Requires-Dist: litellm (>=1.30,<2.0) ; extra == "llm" or extra == "all"
Requires-Dist: loguru (>=0.7,<0.8)
Requires-Dist: numpy (>=1.26,<2.0)
Requires-Dist: openai (>=1.12,<2.0) ; extra == "llm" or extra == "all"
Requires-Dist: openpyxl (>=3.1,<4.0) ; extra == "excel" or extra == "all"
Requires-Dist: pandas (>=2.2,<3.0)
Requires-Dist: polars (>=0.20,<0.21)
Requires-Dist: psycopg[binary] (>=3.1,<4.0) ; extra == "databases" or extra == "all"
Requires-Dist: pyarrow (>=16.0.0) ; extra == "databases" or extra == "all"
Requires-Dist: pydantic (>=2.6,<3.0)
Requires-Dist: pymongo (>=4.6,<5.0) ; extra == "mongo" or extra == "all"
Requires-Dist: requests (>=2.31,<3.0) ; extra == "remote" or extra == "all"
Requires-Dist: rich (>=13.7,<14.0)
Requires-Dist: scikit-learn (>=1.4,<2.0) ; extra == "ml" or extra == "all"
Requires-Dist: sqlalchemy (>=2.0,<3.0)
Requires-Dist: tiktoken (>=0.6,<0.7) ; extra == "llm" or extra == "all"
Requires-Dist: xlrd (>=2.0,<3.0) ; extra == "excel" or extra == "all"
Project-URL: Documentation, https://google.github.io/autonomous-data-quality-agent/
Project-URL: Homepage, https://github.com/google/autonomous-data-quality-agent
Project-URL: Repository, https://github.com/google/autonomous-data-quality-agent
Description-Content-Type: text/markdown

# ADQA: Autonomous Data Quality Agent

[![PyPI version](https://img.shields.io/pypi/v/adqa.svg)](https://pypi.org/project/adqa/)
[![Python versions](https://img.shields.io/pypi/pyversions/adqa.svg)](https://pypi.org/project/adqa/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**ADQA** is an intelligent, autonomous agent designed for high-performance data quality inspection, risk detection, and automated remediation. Built for the era of **Data-centric AI**, ADQA helps you ensure your models are trained on clean, reliable, and safe data.

## 🚀 Vision

Automating data quality and risk inspection to make the work of data scientists and engineers easier, safer, and more productive. ADQA combines a powerful Python library for automated pipelines with a high-performance Rust-based TUI for interactive observability.

## 📚 Documentation

For full documentation, architecture details, and API reference, visit:
**[google.github.io/autonomous-data-quality-agent](https://google.github.io/autonomous-data-quality-agent/)**

## ✨ Key Features

- **Multi-Source Ingress:** Support for CSV, Parquet, Excel, SQL databases, S3, Airbyte, and more.
- **Intelligent Profiling:** Structural, relational, and ML-based profiling with deterministic caching.
- **Risk Inspection:** 
    - **PII Detection:** Automated identification of emails, phone numbers, credit cards, SSNs, etc.
    - **Bias Detection:** Spotting extreme class imbalances in sensitive demographic columns.
    - **Quality Checks:** Missing values, outliers, skewness, pattern violations, and more.
- **Autonomous Remediation:** 
    - **Advisory Mode:** Observe what would happen without modifying data.
    - **Automatic Mode:** Heal data autonomously (impute, drop, clip, mask).
    - **Human-in-the-Loop:** Interactive approval of proposed fixes via CLI or TUI.
- **Full Traceability:** Detailed execution traces and data lineage for every decision made by the agent.

## 📦 Installation

```bash
pip install adqa
```

## 🛠 Usage

### Command Line Interface (CLI)

ADQA comes with a powerful CLI for quick inspections:

```bash
# Basic advisory analysis
adqa analyze data.csv

# Run in automatic remediation mode and save results
adqa analyze data.csv --mode automatic --output cleaned_data.csv

# Human-in-the-loop mode
adqa analyze data.csv --mode human_in_loop
```

### Python API

Integrate ADQA into your existing data pipelines:

```python
from adqa import ADQA, DataSource, ADQAConfig

# Load data
source = DataSource.csv("path/to/data.csv")

# Configure agent
config = ADQAConfig(execution_mode="automatic")

# Run analysis
agent = ADQA(data_source=source, config=config)
result = agent.analyze()

# Get the healed dataframe
df_cleaned = result.dataframe

# See the summary
print(result.summary())
```

## 🖥 Rust TUI

For an interactive, real-time view of the ADQA agent's reasoning and trace events, use the ADQA TUI:

```bash
cd adqa-tui
cargo run
```

## 📄 License

MIT License. See [LICENSE](LICENSE) for details.

