---
name: ao-validation
description: "Pre-commit and pre-merge validation checks. Use before committing changes or declaring work complete to ensure all quality gates pass."
category: core
invokes: [ao-state, ao-baseline, ao-task, ao-api-review]
invoked_by: [ao-implementation, ao-critical-review, ao-git]
state_files:
  read: [constitution.md, baseline.md, focus.json, issues/*.md]
  write: [focus.json, issues/*.md]
references: [.ao/references/confidence.md]
---

# Validation Workflow

> **Confidence Reference**: This skill uses canonical confidence definitions from [.ao/references/confidence.md](.ao/references/confidence.md). Validation tier is determined by confidence level.

**Works with or without `aoc` CLI installed.** Issue tracking can be done via direct file editing.

## Purpose

Ensure all quality gates pass before committing changes or declaring work complete. This skill consolidates all validation checks into a single, consistent procedure.

## Validation Commands (from constitution)

```bash
# Example commands — read actual commands from .agent/ops/constitution.md
build: npm run build          # or: uv run python -m build
lint: npm run lint            # or: uv run ruff check .
test: npm run test            # or: uv run pytest
format: npm run format        # or: uv run ruff format .
```

## Issue Operations After Validation (ao)

| Operation | Command |
|-----------|---------|
| Create regression issue | `ao --yes issue add "Regression: ..." priority:high type:BUG` |
| Update issue status | `ao --yes issue patch <ID> status:done` |
| List blocking issues | `ao ls --status blocked` |

### Example: Post-Validation Issue Creation

```bash
ao --yes issue add "New test failure: UserService.login" priority:high type:BUG
ao --yes log add <ID> "Regression detected during validation"
```

## API Detection

**Before running validation, check if project contains APIs:**

```yaml
api_indicators:
  - OpenAPI/Swagger spec (openapi.yaml, swagger.json, openapi.json)
  - API framework patterns (FastAPI, Flask, Express, ASP.NET controllers)
  - Route decorators (@app.route, @router.get, [HttpGet], etc.)
```

**If API detected during Tier 3 validation:**
1. Note: "API endpoints detected"
2. After standard validation, invoke `ao-api-review` for contract alignment check
3. Include API review findings in validation report

## When to Use

- Before any git commit
- Before declaring a task complete
- Before critical review
- After recovery actions
- On explicit user request

## Preconditions

- `.agent/ops/constitution.md` exists with confirmed commands
- `.agent/ops/baseline.md` exists for comparison

## Validation Tiers

### Low-Confidence Tier Enforcement (HARD RULE)

**For `confidence: low` issues, Tier 1 (quick) validation is FORBIDDEN.**

| Confidence | Minimum Tier | Baseline Required | Notes |
|------------|--------------|-------------------|-------|
| LOW | Tier 2 (HARD) | YES (mandatory) | Cannot use quick-only validation |
| NORMAL | Tier 2 (soft) | Recommended | Can use Tier 1 for quick checks |
| HIGH | Tier 1 (ok) | Optional | Quick validation acceptable |

**Enforcement check at validation start:**
```
IF issue.confidence == 'low':
    IF requested_tier == 1:
        ERROR: "Low-confidence issues require Tier 2+ validation"
        Upgrade to Tier 2 automatically
    IF NOT baseline_exists:
        ERROR: "Low-confidence issues require baseline for comparison"
        BLOCK until baseline is captured
```

### Tier 1: Fast Checks (always run)

Run duration: < 30 seconds

**⚠️ NOT VALID for `confidence: low` issues — automatically upgrade to Tier 2**

1. **Syntax validation**: Files parse without errors
2. **Lint (fast mode)**: Style and obvious issues
3. **Type check** (if applicable): Static type errors
4. **Format check**: Code formatting consistent

### Tier 2: Standard Checks (before commit)

Run duration: < 5 minutes

1. All Tier 1 checks
2. **Unit tests**: Fast, isolated tests
3. **Build**: Project compiles/builds successfully
4. **Lint (full)**: Complete lint analysis

### Tier 3: Comprehensive Checks (before merge/complete)

Run duration: varies (can be slow)

1. All Tier 2 checks
2. **Integration tests**: Component interaction tests
3. **Coverage check**: Ensure coverage thresholds met
4. **Security scan** (if configured): Vulnerability detection
5. **Documentation**: Verify docs are updated

### Tier 4: Preview Validation (when applicable)

**Run when issue has linked preview (`.agent/ops/issues/references/{issue_id}-preview.md`).**

Run duration: < 1 minute (file comparison)

1. **Load preview snapshot** from issue reference file
2. **Get actual git diff** vs baseline or branch point
3. **Compare file lists**:
   - Expected files in preview
   - Actual files changed
   - Flag: extra files, missing files
4. **Compare change scope**:
   - Expected line counts from preview
   - Actual line counts per file
   - Flag: >10% deviation (strict mode)
5. **Check security-sensitive files**:
   - Match against path patterns: `**/auth/**`, `**/security/**`, `**/*secret*`, `**/*cred*`, `**/*token*`
   - Match against content keywords: password, api_key, secret, credential, auth, permission
   - Flag: security-sensitive file touched
6. **Generate discrepancy report**
7. **HARD STOP if discrepancies found** — cannot proceed until resolved

**Preview Validation Procedure:**

```
function validate_preview(issue_id):
    preview_path = .agent/ops/issues/references/{issue_id}-preview.md
    
    IF NOT file_exists(preview_path):
        SKIP — no preview to validate
        RETURN PASS
    
    // Parse validation snapshot from preview file
    snapshot = parse_validation_snapshot(preview_path)
    
    // Get actual changes
    actual_files = git_diff_files()
    
    // Compare
    extra = actual_files - snapshot.expected_files
    missing = snapshot.expected_files - actual_files
    
    // Scope check
    scope_violations = []
    FOR file IN actual_files ∩ snapshot.expected_files:
        expected = snapshot.scope_estimates[file]
        actual = count_lines_changed(file)
        IF abs(actual - expected) / expected > 0.10:
            scope_violations.append((file, expected, actual))
    
    // Security check
    security_flags = []
    FOR file IN actual_files:
        IF matches_security_pattern(file) OR contains_security_keyword(file):
            security_flags.append(file)
    
    // Result
    IF extra OR missing OR scope_violations OR (security_flags AND NOT all_approved):
        RETURN BLOCKED, generate_report(...)
    ELSE:
        RETURN PASS
```

**Integration with other tiers:**

| Validation | When Preview Exists |
|------------|---------------------|
| Tier 1 (quick) | Skip preview validation (too light) |
| Tier 2 (standard) | Include preview validation |
| Tier 3 (comprehensive) | Include preview validation with full report |

## Procedure

### Step 0: Check for Preferences

Check for `preferences.md` in this skill's directory.

**Path**: `.ao/skills/ao-validation/preferences.md`

```
IF file_exists(PREFERENCES_PATH):
    Parse frontmatter:
    - validation_tier (tier1/tier2/tier3) — default validation level
    - allow_baseline_skip (yes/no) — allow skipping baseline comparison
    - auto_fix_lint (yes/no) — auto-fix lint issues before reporting
    - fail_fast (yes/no) — stop on first failure
    Log: "✓ Loaded validation preferences"
ELSE:
    Log: "No preferences found, using defaults"
    Proceed to Quick Validation
```

### Quick Validation (Tier 1)

```
1. Run lint command (fast mode if available)
2. Run type check command (if applicable)
3. Check for syntax errors in changed files
4. Report: PASS / FAIL with details
```

### Standard Validation (Tier 2)

```
1. Run Tier 1 checks
2. Run build command from constitution
3. Run unit test command from constitution
4. Compare results to baseline
5. Report: PASS / FAIL / REGRESSION
```

### Comprehensive Validation (Tier 3)

```
1. Run Tier 2 checks
2. Run full test suite
3. Run coverage analysis
4. Run security checks (if configured)
5. Verify documentation updated
6. Compare all results to baseline
7. Report: PASS / FAIL / REGRESSION with full details
```

### Preferences Creation Offer

After successful validation run, if `preferences.md` doesn't exist:

> "Would you like me to save your validation preferences?"

**If yes**, interview for:
1. **Default Tier**: "What validation tier should be the default?" (tier1/tier2/tier3, default: tier2)
2. **Baseline Skip**: "Allow skipping baseline comparison?" (yes/no, default: no)
3. **Auto-fix Lint**: "Auto-fix lint issues before reporting?" (yes/no, default: yes)
4. **Fail Fast**: "Stop on first failure?" (yes/no, default: no)

**Save to** `.ao/skills/ao-validation/preferences.md`

## Validation Report Format

```markdown
## Validation Report - [timestamp]

### Summary
- Tier: [1|2|3]
- Result: [PASS|FAIL|REGRESSION]
- Duration: [time]

### Checks Performed

| Check | Status | Details |
|-------|--------|---------|
| Lint | ✅ PASS | 0 errors, 2 warnings (baseline: 2) |
| Build | ✅ PASS | Exit code 0 |
| Tests | ⚠️ REGRESSION | 1 new failure (see below) |

### Failures (if any)

#### Test Failure: test_feature_x
- File: tests/test_feature.py:42
- Error: AssertionError: expected X, got Y
- Baseline: PASS (new regression)

### Warnings (if any)

- lint: unused variable 'foo' in file.py:10 (pre-existing)

### Recommendation

[PROCEED | FIX REQUIRED | INVESTIGATE]
```

## Baseline Comparison Rules

### New Finding Categories

| Category | Action |
|----------|--------|
| New error | **BLOCK** - must fix before proceeding |
| New warning | **INVESTIGATE** - fix or document why acceptable |
| New test failure | **BLOCK** - must fix or prove pre-existing |
| Improved (fewer issues) | **PASS** - note improvement |
| Same as baseline | **PASS** - no change |

### Handling Regressions

1. Identify if regression is from agent's changes
2. If yes: fix before proceeding
3. If no (pre-existing): document and escalate as task
4. Never ignore regressions silently

## Integration Points

### With ao-git

Before committing:
```
1. Run Tier 2 validation
2. If PASS: proceed with commit
3. If FAIL: abort commit, report issues
```

### With ao-critical-review

During review:
```
1. Run Tier 3 validation
2. Include validation report in review
3. Block completion if FAIL or REGRESSION
```

### With ao-implementation

After each step:
```
1. Run Tier 1 validation (fast feedback)
2. After final step: run Tier 2 validation
```

## Quality Gate Configuration

### Confidence-Based Coverage Thresholds

**Coverage requirements vary by confidence level:**

| Confidence | Line Coverage | Branch Coverage | Gate Type |
|------------|---------------|-----------------|----------|
| LOW | ≥90% on changed code | ≥85% on changed code | HARD (blocks) |
| NORMAL | ≥80% on changed code | ≥70% on changed code | SOFT (warning) |
| HIGH | Tests pass | N/A | None |

**During Tier 3 validation, check coverage against confidence threshold:**

```
🎯 COVERAGE VALIDATION — {CONFIDENCE} Confidence

| Metric | Required | Actual | Status |
|--------|----------|--------|--------|
| Line coverage | ≥{threshold}% | {actual}% | {PASS/FAIL} |
| Branch coverage | ≥{threshold}% | {actual}% | {PASS/FAIL} |

{If FAIL for LOW confidence:}
⛔ COVERAGE THRESHOLD NOT MET — Cannot proceed

Options:
1. Add more tests to reach threshold
2. Document why threshold is unachievable (requires justification)
```

### Constitution-Based Configuration

Read from `.agent/ops/constitution.md`:

```markdown
## Quality gates
- lint_must_pass: true | false
- build_must_pass: true
- tests_must_pass: true
- coverage_threshold: 80% | none
- allow_warnings: true | false
- security_scan: true | false
```

If not configured, defaults:
- lint_must_pass: true
- build_must_pass: true  
- tests_must_pass: true
- coverage_threshold: none
- allow_warnings: true
- security_scan: false

## Commands

All validation commands MUST come from constitution. Never guess or assume commands.

```markdown
## From constitution:
- build: [constitution build command]
- lint: [constitution lint command]
- test: [constitution test command]
- format: [constitution format command]
```

## Output

Update `.agent/ops/focus.json` with validation results:
```markdown
## Just did
- Ran Tier 2 validation: PASS
  - lint: 0 errors, 2 warnings (baseline match)
  - build: success
  - tests: 45 pass, 0 fail
```

## Issue Discovery After Validation

**After validation, invoke `ao-task` discovery procedure for new findings:**

1) **Collect regressions and new issues:**
   - New test failures → `BUG` (high/critical)
   - New lint errors → `BUG` (medium)
   - New warnings → `CHORE` (low)
   - Coverage drops → `TEST` (medium)
   - Security findings → `SEC` (high/critical)

2) **Present to user:**
   ```
   📋 Validation found {N} new issues vs baseline:
   
   Critical:
   - [BUG] New test failure: UserService.login
   
   High:
   - [SEC] New security warning from npm audit
   
   Medium:
   - [BUG] 2 new lint errors in PaymentController
   
   Create issues to track these? [A]ll / [S]elect / [N]one
   
   Note: These MUST be fixed before commit/merge.
   ```

3) **After creating issues:**
   ```
   Created {N} issues. These block commit/merge.
   
   1. Start fixing highest priority (BUG-0024@abc123)
   2. View all blocking issues
   3. Abort current work
   ```

```

## Preferences Update Trigger

**Trigger phrases:**
- "Update my validation preferences"
- "What are my validation preferences?"
- "Reset validation preferences"
- "preferences" (during validation)

**Display format:**
```
📋 Current Validation Preferences:
- Default Tier: {value}
- Baseline Skip: {value}
- Auto-fix Lint: {value}
- Fail Fast: {value}

Update? [Y]es / [N]o
```
