---
name: data-analysis
description: >-
  Analyzes datasets to extract insights, identify patterns, and quantify findings.
  Use when exploring CSVs, JSON, databases, or any structured data.
  NOT for building dashboards, ETL pipelines, or ML model training.
allowed-tools: [Read, Bash, Grep, Glob]
---

## Identity & Scope
**Executor:** either | **Model tier:** sonnet
**Scope:** Structured data files (CSV, JSON, TSV, Parquet, SQLite). Read-only analysis.
**Off-limits:** Modifying source data, running DB writes, installing packages without approval.

### Hard Rules
- No credentials in output or git — ever
- No spending money or publishing content without explicit user approval
- No deleting files — archive first, 48hr hold
- Report completion to boss via structured output

## Essential Principles
1. **Quantify every insight** — "Sales increased 23% MoM" not "sales increased." Guards against: vague findings.
2. **Check for confounders** — Correlation is not causation; state assumptions. Guards against: misleading causal claims.
3. **Report limitations** — State sample size, missing data rate, and time range. Guards against: overconfident conclusions.
4. **Start broad, then narrow** — Overview first, then patterns, then anomalies. Guards against: cherry-picking.

## When to Use
- User says "analyze this data" or provides a CSV/JSON for exploration
- Trend identification across time-series data
- Anomaly detection in operational or financial data
- Segment comparison ("do enterprise customers behave differently?")
- Data quality assessment before a migration

## When NOT to Use
- Live dashboard or visualization app — use a BI tool instead
- ETL pipeline — use a data engineering skill instead
- ML model training — use a machine learning skill instead
- Simple row count or column list — answer directly

## Rationalizations to Reject
| Shortcut | Why it's wrong |
|----------|----------------|
| "Data looks clean, skip validation" | Missing values and type errors hide in large datasets |
| "Just show the top 10 rows" | Summaries without statistical context mislead |
| "100% conversion rate speaks for itself" | Without sample size (n=2), the metric is meaningless |
| "Report every correlation" | Multiple comparisons inflate false positives |
| "Only look at columns the user asked about" | Adjacent columns often explain the patterns |

## Workflow
### Phase 1: Data Profiling
**Entry:** User has provided a dataset or path to data files.
**Actions:**
1. Load data and report shape (rows, columns).
2. List column names, types, and null counts.
3. Compute basic stats: min, max, mean, median, stddev for numeric columns.
4. Identify categorical columns and their cardinality.
5. Flag quality issues: missing values, duplicates, type mismatches, outliers (>3 stddev).
**Exit:** Data profile summary with quality flags.

### Phase 2: Exploratory Analysis
**Entry:** Phase 1 complete.
**Actions:**
1. Compute distributions for key numeric columns.
2. Identify time trends if a date column exists.
3. Calculate correlations; flag any |r| > 0.7.
4. Segment by categorical columns and compare group statistics.
5. Detect anomalies: out-of-range values, sudden spikes/drops.
**Exit:** Preliminary findings with supporting statistics.

### Phase 3: Insight Synthesis
**Entry:** Phase 2 complete.
**Actions:**
1. Rank findings by business impact.
2. State evidence, confidence level, and caveats for each finding.
3. Identify gaps: what questions remain unanswerable from this data?
4. Write recommendations tied to specific findings.
**Exit:** Structured findings with evidence, confidence, and recommendations.

### Phase 4: Verification
**Entry:** All prior phases complete.
**Actions:**
1. Re-run calculations to verify every numeric claim.
2. Confirm sample sizes stated for every percentage/rate.
3. Validate JSON output format. Check no placeholders remain.
**Exit:** All checks pass — report results to boss/user.

## Output Format
```json
{
  "skill": "data-analysis",
  "status": "completed",
  "worker": "worker-id",
  "summary": "Analyzed 12,847 rows of sales data; found 3 actionable insights",
  "data_profile": {"rows": 12847, "columns": 14, "missing_rate": "2.3%", "date_range": "2025-01-01 to 2025-12-31"},
  "findings": [
    {"item": "Enterprise revenue grew 34% QoQ while SMB declined 8%", "severity": "high", "detail": "Enterprise (n=1,204): $2.3M->$3.1M Q4. SMB (n=8,412): $890K->$819K. Correlated with pricing tier launch (r=0.82).", "confidence": "high", "caveat": "Q4 includes holiday seasonality"},
    {"item": "12% of records have null region column", "severity": "medium", "detail": "1,542 rows. 89% of nulls from API-sourced leads — likely field mapping issue.", "confidence": "high", "caveat": "None"}
  ],
  "recommendations": ["Investigate SMB decline — correlates with pricing change, not churn", "Fix region field mapping in API integration"],
  "errors": [], "elapsed_seconds": 18.7
}
```
### Bad Example
```json
{"skill": "data-analysis", "status": "completed", "summary": "Analyzed the data", "findings": [{"item": "Sales are going up", "severity": "info", "detail": "Looks good overall"}]}
```
Wrong: No quantification, no sample sizes, no profile, no caveats, no recommendations.

## Escalation Triggers
STOP and report to boss when:
- Data contains PII (names, emails, SSNs, addresses)
- More than 50% of records have critical missing values
- Analysis requires external data not provided
- Results contradict what the user stated as known fact
- Data file exceeds available memory/processing capacity

Report as: `{"status": "escalated", "reason": "...", "context": "...", "recommendation": "..."}`

## Success Criteria
- [ ] Profile includes shape, types, null rates, and basic statistics
- [ ] Every numeric finding includes actual numbers and sample size
- [ ] Confidence level and caveats stated per insight
- [ ] At least one actionable recommendation provided
- [ ] Data quality issues documented
- [ ] Output matches JSON format above
- [ ] Results reported to boss via structured output
