---
# Stage identity per spec Section 6.1
stage: 2
stage_id: "02"
title: "File-Level Analysis"
workflow: understand
estimated_tokens: 4000

# Context specification
context:
  static:
    persona: "codebase analyst performing file-level understanding and optional docstring updates"
    purpose: "Analyze each stale file, optionally update docstrings, then store understanding in AI cache"
  dynamic:
    - key: mode
      source: cli_arg
      default: "standard"
    - key: max_agents
      source: cli_arg
      default: 10
    - key: docstrings
      source: cli_arg
      default: "false"
    - key: stale_files
      source: stage_1_output
      required: true

# Tier specification for sub-agents
tier:
  name: fast
  hint: "Simple, repetitive file analysis task - use fastest available model"

# Pre-validation checks
pre_validation:
  - check: stale_files_exist
    description: "Stale files list is available from Stage 1"
    severity: error

# Execution flow with step types
execution_flow:
  - step: 1
    type: action
    name: group_by_depth
    instruction: "Group stale files by directory depth (deepest first)"
    output_variable: depth_groups
  - step: 2
    type: loop
    name: process_depth_level
    instruction: "For each depth level, process all files at that level"
    items: depth_groups
    sub_steps:
      - step: 2a
        type: parallel
        name: analyze_files
        instruction: "Analyze files in parallel (max {max_agents} concurrent)"
      - step: 2b
        type: action
        name: collect_results
        instruction: "Collect results from completed analyses"
  - step: 3
    type: action
    name: generate_report
    instruction: "Generate completion report with statistics"

# Guardrails
guardrails:
  enforcement: active
  allowed_tools:
    - civyk-repoix
    - Read
    - Grep
    - Glob
    - Bash
    - Edit  # only permitted when docstrings=true, for docstring updates only
  forbidden_actions:
    - "Delete any files"
    - "Use Edit tool when docstrings=false"
    - "Modify function logic or implementation"
    - "Change variable names, types, or signatures"
    - "Add new functions or classes"
  modifications_allowed:
    - "Add or update docstrings only (when docstrings=true)"
    - "Preserve existing docstring style"

# Post-validation checks
post_validation:
  - check: all_files_processed
    description: "All stale files were analyzed (or errors logged)"
    severity: warning
  - check: understanding_stored
    description: "Understanding was stored for processed files"
    severity: error

# Next stage - continues to module aggregation unless mode=quick
next:
  stage: 3
  stage_id: "03"
  auto_continue: true
  terminal_conditions:
    - condition: "mode == 'quick'"
      description: "Quick mode stops after file-level analysis (docstrings handled in this stage)"
version: 1.0.0
---
# Stage 2: File-Level Analysis

Analyze each stale file from Stage 1 and store understanding in the AI cache.

## Prerequisites

- Stage 1 completed with stale files list
- civyk-repoix available (mode determined in Stage 1)

## Critical Rules

| Rule | Action |
|------|--------|
| Depth-first | **MUST** process deepest directories first |
| Re-check staleness | **MUST** verify file still stale before processing |
| Docstrings before store | **MUST** write docstrings (if `--docstrings`) before calling `understand(action="store")` |
| Store always | **MUST** call `understand(action="store")` after each file (after any edits) |
| Handle errors | **MUST** continue on individual file failures |
| Retry on failure | **MUST** retry failed files up to 3 times |
| Save checkpoint | **MUST** save progress after each depth level |

---

## Retry Logic

For transient failures (timeouts, rate limits), retry before marking as failed:

```text
MAX_RETRIES = 3
RETRY_DELAY = 2  # seconds

FOR file IN stale_files
    FOR attempt IN 1..MAX_RETRIES
        TRY
            result = analyze_single_file(file)
            IF success THEN
                BREAK  # Success, move to next file
            END IF
        CATCH error
            IF attempt < MAX_RETRIES THEN
                OUTPUT "[RETRY {attempt}/{MAX_RETRIES}] {file_path}: {error}"
                WAIT(RETRY_DELAY * attempt)  # Exponential backoff
            ELSE
                OUTPUT "[FAILED] {file_path}: {error} (after {MAX_RETRIES} attempts)"
                errors.append({file, error, recoverable=true})
                CONTINUE  # to next file
            END IF
        END CATCH
    END FOR
END FOR
```

**Retryable errors:**
- Timeout errors
- Rate limit errors (429)
- Temporary network issues
- Sub-agent spawn failures

**Non-retryable errors:**
- Parse errors (file is malformed)
- Permission errors
- File not found (deleted during processing)

---

## Checkpointing & Resume

### Checkpoint Structure

```text
understand_checkpoint_stage2 = {
    "stage": 2,
    "status": "in_progress",
    "current_depth": <N>,
    "depths_completed": [<list>],
    "files_processed": [<list>],
    "files_remaining": [<list>],
    "errors": [<list>],
    "stats": {
        "processed_count": <N>,
        "skipped_count": <N>,
        "error_count": <N>
    }
}
```

### Save Checkpoint After Each Depth Level

```text
# After processing each depth level:
understand(
    action="store",
    scope="project",
    target="understand-file-checkpoint",
    purpose="Understand workflow checkpoint for Stage 2",
    importance="high",
    key_points=[
        "stage: 2",
        "current_depth: {depth}",
        "processed: {processed_count}",
        "remaining: {remaining_count}"
    ],
    gotchas=["Resume with: civyk-hatch understand --resume"],
    analysis="Stage 2 in progress. Depth {depth} complete. {processed_count}/{total} files processed."
)
```

### Resume from Checkpoint

On Stage 2 start, check for existing checkpoint:

```text
understand(action="recall", target="understand-file-checkpoint")

IF found AND key_points contains "stage: 2" THEN
    # Resume from checkpoint
    current_depth = extract("current_depth")
    files_processed = extract("files_processed")

    # Skip already processed depths
    FOR depth IN range(max_depth, current_depth, -1)
        SKIP "Already processed"
    END FOR

    # Continue from current depth
    OUTPUT "Resuming from depth {current_depth}. {processed} files already done."
END IF
```

### Graceful Interrupt Handling

If Ctrl+C or interrupt detected:
1. Complete current file (if possible)
2. Save checkpoint with current progress
3. Report resume instructions:

```text
=== WORKFLOW INTERRUPTED ===

Progress saved to checkpoint.
Processed: {processed_count}/{total_count} files
Current depth: {depth}

To resume: civyk-hatch understand --resume
```

---

## Tier Specification

```text
Tier: fast
Hint: Simple, repetitive file analysis task
```

This stage uses the `fast` tier for file analysis sub-agents. The orchestrator maps this to the appropriate model based on availability (e.g., Haiku for Claude, GPT-5 mini for OpenAI).

---

## Step 1: Group Files by Directory Depth

Organize stale files by their directory depth to enable depth-first processing:

```text
Depth 3 (deepest):
  - src/core/utils/helpers.py
  - src/api/v2/handlers.py

Depth 2:
  - src/core/engine.py
  - src/api/routes.py

Depth 1:
  - src/main.py
  - src/config.py

Depth 0:
  - setup.py
  - pyproject.toml
```

**Process order:** Deepest first (3 -> 2 -> 1 -> 0)

---

## Step 2: Process Each Depth Level

For each depth level, process all files at that level:

### Step 2a: Parallel File Analysis

Process files in parallel batches (max `{max_agents}` concurrent):

```text
# For each file in current depth level:
FOR batch IN chunk(depth_files, max_agents)
    PARALLEL FOR file IN batch
        analyze_single_file(file)
    END PARALLEL FOR

    # IMPORTANT: Report progress after each batch
    REPORT "Processed {completed}/{total} files at depth {depth}"
END FOR
```

**CRITICAL: Real-Time Status Updates**

The main orchestrator (you) must emit status updates as agents complete - do NOT spawn separate status agents.

**After EACH sub-agent returns:**
```text
# Immediately when a sub-agent completes:
OUTPUT: "[{completed}/{total}] [ok] {file_path} - {status}"

# Example outputs:
"[1/45] [ok] src/core/engine.py - stored (critical)"
"[2/45] [ok] src/core/utils.py - stored (medium)"
"[3/45] [err] src/api/broken.py - error: parse failed"
```

**After each batch completes:**
```text
# Update todo for visual progress bar effect
TodoWrite: "Analyzing files: {completed}/{total} complete ({pct}%)"

# Emit batch summary
OUTPUT: "Batch {N} complete: {batch_size} files in {duration}s"
```

**Why orchestrator handles status (not sub-agents):**
- Orchestrator receives results as they arrive
- No coordination overhead
- Natural aggregation point for progress
- Sub-agents focus purely on analysis

**CRITICAL: Checkpointing**

After processing each depth level:
1. Report: "Depth {N} complete: {count} files processed"
2. If interrupted, next run can resume from last completed depth

### Step 2b: Single File Analysis

For each file, perform the following analysis:

#### 2b.1: Verify Still Stale

```text
understand(action="recall", target="<file_path>")

IF found=true AND fresh=true THEN
    SKIP "Already processed by another agent"
    RETURN "Skipped (cache hit)"
END IF
```

#### 2b.2: Get File Structure

```text
# Get symbols without reading full file (saves tokens)
file(action="symbols", path="<file_path>", include_private=true)

# Get imports to understand dependencies
file(action="imports", path="<file_path>")
```

#### 2b.3: Read File Content

```text
# Read file content for analysis
Read(file_path="<file_path>")
```

#### 2b.4: Analyze File

Analyze the file content to extract:

| Aspect | Description |
|--------|-------------|
| Purpose | What does this file do? Main responsibility |
| Exports | Public classes, functions, constants |
| Dependencies | What does it import/require? |
| Key Logic | Critical algorithms or patterns |
| Entry Points | Main functions, handlers, routes |

#### 2b.5: Determine Importance

| Importance | Criteria |
|------------|----------|
| `critical` | Entry points, core business logic, configuration |
| `high` | Services, handlers, models, utilities used widely |
| `medium` | Helpers, internal modules, domain-specific code |
| `low` | Scripts, tests, examples, generated code |

#### 2b.6: Update Docstrings (only when `docstrings=true`)

Skip this step entirely when `docstrings=false`.

```text
IF docstrings == true THEN
    # Identify missing or outdated docstrings based on the analysis above
    FOR each public class, function, or method in file
        IF docstring is missing OR outdated THEN
            Edit(file_path="<file_path>", ...)  # add/update docstring only
        END IF
    END FOR
END IF
```

**Rules:**
- Only add or update docstrings — never touch logic, signatures, or structure
- Preserve existing docstring style (Google, NumPy, reStructuredText, plain)
- After all edits are applied, the file on disk must reflect the final state before the next step

---

#### 2b.7: Store Understanding

```text
understand(
    action="store",
    scope="file",
    target="<file_path>",
    purpose="<one-line description of file purpose>",
    importance="<critical|high|medium|low>",
    key_points=[
        "<main class/function>",
        "<key exports>",
        "<notable patterns>"
    ],
    gotchas=[
        "<potential issues>",
        "<non-obvious behavior>",
        "<required preconditions>"
    ],
    analysis="<detailed analysis - 2-3 sentences describing the file's role,
              key components, and relationships to other parts of the codebase>"
)
```

**Example:**

```text
understand(
    action="store",
    scope="file",
    target="src/core/engine.py",
    purpose="Core processing engine for data transformation pipeline",
    importance="critical",
    key_points=[
        "Main class: DataEngine",
        "Entry point: process() method",
        "Handles: validation, transformation, output"
    ],
    gotchas=[
        "Not thread-safe - use ProcessPool for parallelism",
        "Requires initialized ConfigLoader before use",
        "transform() may raise ValidationError on malformed input"
    ],
    analysis="DataEngine orchestrates the data transformation pipeline.
              The process() method validates input via InputValidator,
              applies transformations from TransformRegistry, and writes
              output via OutputWriter. Dependencies: config, utils, validators."
)
```

---

## Step 3: Handle Errors Gracefully

Individual file failures should not stop the workflow:

```text
FOR file IN stale_files
    TRY
        result = analyze_single_file(file)
        processed.append(result)
    CATCH error
        errors.append({
            "file": file,
            "error": str(error),
            "recoverable": true
        })
        CONTINUE  # Don't fail entire workflow
    END TRY
END FOR
```

---

## Step 4: Generate Completion Report

```text
=== UNDERSTAND WORKFLOW - FILE ANALYSIS COMPLETE ===

Duration: {duration}
Mode: {mode}

=== PROCESSING SUMMARY ===
Files:
|-- Processed: {processed_count}
|-- Skipped (cache hit): {skipped_count}
|-- Errors: {error_count}
|-- Total: {total_files}

Processing by depth:
|-- Depth {max_depth}: {depth_n_count} files
|-- Depth {max_depth-1}: {depth_n1_count} files
|-- ...
|-- Depth 0: {depth_0_count} files

=== CACHE UPDATES ===
Understanding stored:
|-- New entries: {new_entries}
|-- Updated entries: {updated_entries}

Coverage:
|-- Files with understanding: {covered_files}/{total_files} ({coverage_pct}%)

=== PERFORMANCE METRICS ===
Sub-agent usage:
|-- Total agents spawned: {agent_count}
|-- Max concurrent: {max_concurrent}
|-- Avg file processing time: {avg_time}

Token usage (estimated):
|-- Input tokens: {input_tokens}K
|-- Output tokens: {output_tokens}K
|-- Total tokens: {total_tokens}K

=== CHECKPOINT STATUS ===
Checkpoint: Cleared (stage complete)
{IF mode in ["standard", "deep"]:}
Next stage: 03-module-aggregation
{ELSE:}
Workflow complete (quick mode)
{END IF}

{IF errors:}
=== ERRORS (will retry on next run) ===
{FOR error in errors:}
|-- {error.file}: {error.error}
{END FOR}
{END IF}
```

---

## Understanding Schema Reference

### File-Level Understanding

```python
understand(
    action="store",          # Required: "store" action
    scope="file",           # Required: "file" for file-level
    target="<file_path>",   # Required: relative path from repo root
    purpose="<string>",     # Required: one-line purpose description
    importance="<level>",   # Required: critical|high|medium|low
    key_points=["..."],     # Required: list of 2-5 key points
    gotchas=["..."],        # Optional: list of gotchas/warnings
    analysis="<string>"     # Required: detailed analysis (2-3 sentences)
)
```

---

## Exit Conditions

### All Files Processed Successfully

```text
=== FILE-LEVEL ANALYSIS COMPLETE ===

All {processed_count} stale files have been analyzed and cached.

Cache efficiency:
|-- New entries: {new_count}
|-- Cache hits (skipped): {skip_count}
|-- Hit rate: {hit_rate}%

[ok] UNDERSTAND WORKFLOW COMPLETE (quick mode)

Next steps:
- Run 'civyk-hatch understand' again after code changes
- Understanding will be used by sync-docs and other workflows
```

**STOP** - Workflow complete (quick mode terminal).

### Partial Success with Errors

```text
=== FILE-LEVEL ANALYSIS COMPLETE (WITH ERRORS) ===

Processed: {processed_count}/{total_count} files
Errors: {error_count}

Errors (will retry on next run):
{error_list}

[!] Re-run 'civyk-hatch understand' to retry failed files.
```

**STOP** - Workflow complete with warnings.
