---

stage: 3
stage_id: "03"
title: "Module Aggregation"
workflow: understand
estimated_tokens: 3000

context:
  static:
    persona: "codebase analyst synthesizing module-level understanding"
    purpose: "Aggregate file understandings into module/directory summaries"
  dynamic:
    - key: mode
      source: cli_arg
      default: "standard"
    - key: max_agents
      source: cli_arg
      default: 10

tier:
  name: balanced
  hint: "Synthesis task requiring aggregation of multiple file understandings"

pre_validation:
  - check: file_analysis_complete
    description: "Stage 2 completed with file-level understanding cached"
    severity: error

execution_flow:
  - step: 1
    type: action
    name: build_directory_tree
    instruction: "Build hierarchical directory tree from processed files"
    output_variable: dir_tree
  - step: 2
    type: loop
    name: process_directories
    instruction: "Process directories bottom-up (deepest first)"
    items: sorted_dirs
  - step: 3
    type: action
    name: aggregate_module
    instruction: "Recall file understandings and synthesize module summary"
  - step: 4
    type: action
    name: store_module_understanding
    instruction: "Store module-level understanding in cache"

guardrails:
  enforcement: active
  allowed_tools:
    - civyk-repoix
    - Read
    - Grep
    - Glob
    - Bash
  forbidden_actions:
    - "Delete any files"
    - "Modify any source files"
  read_only: true

post_validation:
  - check: modules_processed
    description: "All directories have module-level understanding"
    severity: warning

next:
  stage: 4
  stage_id: "04"
  auto_continue: true
  terminal_conditions:
    - condition: "mode == 'standard'"
      description: "Standard mode stops after module aggregation (docstrings handled in stage 2)"
version: 1.0.0
---

# Stage 3: Module Aggregation

Aggregate file-level understandings into module/directory summaries using bottom-up processing.

## Prerequisites

- Stage 2 completed (file-level analysis done)
- File understandings cached in repoix

## Mode Check

```text
IF mode == "quick" THEN
    SKIP "Mode is 'quick' - skipping module aggregation"
    OUTPUT "Mode is 'quick' - skipping module aggregation"
    EXIT completed
END IF
```

## Critical Rules

| Rule | Action |
|------|--------|
| Bottom-up order | **MUST** process deepest directories first |
| Recall before aggregate | **MUST** recall file understandings before synthesizing |
| Store always | **MUST** call understand(action="store") for each module |
| Handle missing | **CONTINUE** if some file understandings missing |
| Progress tracking | **MUST** report progress after each module processed |
| Save checkpoint | **MUST** save progress after each directory level |

---

## Checkpointing & Resume

### Save Checkpoint After Each Level

```text
# After processing each directory level:
understand(
    action="store",
    scope="project",
    target="understand-module-checkpoint",
    purpose="Understand workflow checkpoint for Stage 3",
    importance="high",
    key_points=[
        "stage: 3",
        "current_level: {level}",
        "modules_completed: {count}",
        "modules_remaining: {remaining}"
    ],
    gotchas=["Resume with: civyk-hatch understand --resume"],
    analysis="Stage 3 in progress. Level {level} complete. {count}/{total} modules aggregated."
)
```

### Resume from Checkpoint

```text
understand(action="recall", target="understand-module-checkpoint")

IF found AND key_points contains "stage: 3":
    current_level = extract("current_level")
    modules_completed = extract("modules_completed")

    # Skip already processed levels
    OUTPUT: "Resuming from level {current_level}. {modules_completed} modules done."
```

---

## Tier Specification

```text
Tier: balanced
Hint: Synthesis task requiring aggregation of multiple inputs
```

This stage uses the `balanced` tier for module synthesis. Aggregating multiple file understandings into a coherent module summary requires reasoning capabilities.

---

## Step 1: Build Directory Tree

Organize processed files into a hierarchical directory structure:

```text
project/
|-- src/
|   |-- core/           <- Process this FIRST (deepest)
|   |   |-- utils/      <- Process this FIRST (deepest)
|   |   |-- engine.py
|   |   `-- config.py
|   |-- api/            <- Process after core/
|   |   |-- routes.py
|   |   `-- handlers.py
|   `-- [MODULE]        <- Aggregate after children done
`-- lib/
    `-- helpers.py
```

**Processing Order:** Bottom-up (deepest directories first)

---

## Step 2: Process Each Directory Level

For each directory level (deepest first):

### Step 2a: Recall File Understandings

```text
FOR each file in directory:
    understand(action="recall", target="<file_path>")

    IF found=true AND fresh=true:
        collect understanding for aggregation
    ELSE:
        log warning: "Missing understanding for <file_path>"
        CONTINUE (don't fail)
```

### Step 2b: Synthesize Module Understanding

Analyze the collected file understandings to identify:

| Aspect | Description |
|--------|-------------|
| Module Purpose | What does this module/directory do as a whole? |
| Key Exports | What are the main public APIs/classes/functions? |
| Internal Structure | How do the files relate to each other? |
| Dependencies | What external modules does this depend on? |
| Entry Points | Which files/functions are the main entry points? |

### Step 2c: Store Module Understanding

```text
understand(
    action="store",
    scope="module",
    target="<directory_path>",
    purpose="<one-line module purpose>",
    importance="<critical|high|medium|low>",
    key_points=[
        "<main components>",
        "<key exports>",
        "<internal relationships>"
    ],
    gotchas=[
        "<initialization requirements>",
        "<circular dependency risks>",
        "<usage constraints>"
    ],
    analysis="<detailed analysis - 2-3 sentences describing the module's
              role, main components, and how files work together>"
)
```

**Example:**

```text
understand(
    action="store",
    scope="module",
    target="src/core",
    purpose="Core processing components for data transformation pipeline",
    importance="critical",
    key_points=[
        "Contains: DataEngine, ConfigLoader, Validators",
        "Entry point: engine.process()",
        "Internal flow: config -> validate -> transform -> output"
    ],
    gotchas=[
        "ConfigLoader must be initialized before DataEngine",
        "utils.py has helper functions used by all other files",
        "Avoid circular imports between engine.py and validators.py"
    ],
    analysis="Core module provides the data transformation pipeline.
              DataEngine orchestrates processing using ConfigLoader for settings
              and Validators for input verification. The utils.py file provides
              shared helpers. Files have a clear dependency hierarchy:
              config <- validators <- engine."
)
```

### Step 2d: Report Progress

**CRITICAL: Real-Time Status Updates**

The main orchestrator (you) must emit status updates as each module agent completes.

**After EACH sub-agent returns:**
```text
# Immediately output status
OUTPUT: "[{completed}/{total}] [ok] {module_path} - {file_count} files aggregated"

# Example outputs:
"[1/12] [ok] src/core/ - 8 files aggregated (critical)"
"[2/12] [ok] src/api/ - 5 files aggregated (high)"
"[3/12] ~ src/utils/ - 3 files aggregated (2 missing)"
```

**After each batch/level completes:**
```text
# Update todo for visual progress
TodoWrite: "Processing modules: {completed}/{total} complete ({pct}%)"

# Emit level summary
OUTPUT: "Level {N} complete: {module_count} modules in {duration}s"
```

**Why orchestrator handles status (not sub-agents):**
- Orchestrator receives results as they arrive
- No coordination overhead
- Natural aggregation point for progress

**CRITICAL: Sub-Agent Usage**

- **DO** use sub-agents for module aggregation (each module is independent)
- **DO** use `balanced` tier (Sonnet) for synthesis tasks
- **DO NOT** use `run_in_background: true` - use foreground agents
- Process modules in parallel batches (max `{max_agents}` concurrent)

```text
FOR batch in chunk(modules, max_agents):
    PARALLEL FOR module in batch:
        Task(
            subagent_type="general-purpose",
            model="sonnet",
            prompt="Aggregate module understanding for {module_path}. Pass file analysis summaries for this module. Expect a module-level summary covering purpose, key abstractions, dependencies, and public interface."
        )
    REPORT: "Batch complete: {completed}/{total} modules"
```

---

## Step 3: Handle Nested Modules

For nested directory structures, process in order:

```text
Level 3 (deepest):
  - src/core/utils/     -> Module understanding stored
  - src/api/v2/         -> Module understanding stored

Level 2:
  - src/core/           -> Aggregates core/utils/ + core files
  - src/api/            -> Aggregates api/v2/ + api files

Level 1:
  - src/                -> Aggregates core/ + api/ + src files

Level 0:
  - project root        -> Handled in Stage 4 (Project Synthesis)
```

---

## Step 4: Generate Completion Report

```text
=== MODULE AGGREGATION COMPLETE ===

Duration: {duration}
Mode: {mode}

=== PROCESSING SUMMARY ===
Modules processed:
|-- Total directories: {dir_count}
|-- Modules created: {module_count}
|-- Skipped (fresh): {skipped_count}

Module hierarchy:
|-- Max depth: {max_depth}
|-- Avg files per module: {avg_files}

Processing by level:
|-- Level {max_level}: {level_n_count} modules
|-- Level {max_level-1}: {level_n1_count} modules
|-- ...
|-- Level 1: {level_1_count} modules

=== CACHE UPDATES ===
Module understanding stored:
|-- New entries: {new_module_entries}
|-- Updated entries: {updated_module_entries}

File understanding referenced:
|-- Cache hits: {file_cache_hits}
|-- Missing (warned): {missing_count}

=== PERFORMANCE METRICS ===
Sub-agent usage:
|-- Total agents spawned: {agent_count}
|-- Max concurrent: {max_concurrent}
|-- Avg module processing time: {avg_time}

Token usage (estimated):
|-- Input tokens: {input_tokens}K
|-- Output tokens: {output_tokens}K

=== CHECKPOINT STATUS ===
Checkpoint: Cleared (stage complete)
{IF mode == "deep":}
Next stage: 04-project-synthesis
{ELSE:}
Workflow complete (standard mode)
{END IF}

{IF warnings:}
=== WARNINGS ===
|-- Missing file understandings: {missing_count}
|-- Tip: Run Stage 2 again to fill gaps
{END IF}
```

---

## Exit Conditions

### Mode Check - Quick Mode

If `mode == "quick"`:

```text
=== SKIPPING MODULE AGGREGATION ===

Mode 'quick' only includes file-level analysis.
Use --mode=standard or --mode=deep for module aggregation.

[ok] UNDERSTAND WORKFLOW COMPLETE (quick mode)
```

**STOP** - Workflow complete for quick mode.

### All Modules Processed

```text
=== MODULE AGGREGATION COMPLETE ===

All {module_count} directories have been aggregated.

[AUTO-CONTINUE] Proceeding to Stage 4: Project Synthesis
```

**CONTINUE** to Stage 4.

---

## [AUTO-CONTINUE]

After module aggregation completes, automatically proceed to Stage 4 for project-level synthesis (unless mode is "standard", which exits after this stage).

## NEXT

```bash
civyk-hatch understand --stage=4 --mode={mode}
```
