# Skill: Add a Retry / Verification Loop

Use when you want an agent to re-run if its output does not meet a quality bar.
The pattern: worker agent produces output → verifier agent checks it → on failure,
retry the worker with feedback.

## How Retry Logic Works in This Framework

`arccrew/orchestrator/graph.py` provides `_make_retry_or_continue()`, a conditional edge
function that reads `verification_result.status` from state:

- `{"status": "pass"}` → route to the next node (continue)
- Anything else → route back to `retry_from` node (retry)

After `max_retries` attempts, it routes forward regardless of status to prevent infinite loops.

Each retry also calls `_make_retry_increment()` which:
1. Increments `retry_count` in state
2. Appends `{"attempt": N, "feedback": "..."}` to `retry_history`

The worker can read `retry_history` to self-correct on subsequent attempts.

## Using the `create_pipeline()` Factory

The simplest way to add retry logic is via the `create_pipeline()` factory in
`arccrew/orchestrator/graph.py`:

```python
from arccrew.orchestrator.graph import create_pipeline
from state import MyState

def create_my_pipeline():
    worker = WorkerAgent()
    verifier = VerifierAgent()

    async def worker_node(state): return await worker.execute(state)
    async def verifier_node(state): return await verifier.execute(state)

    return create_pipeline(
        state_class=MyState,
        nodes={
            "worker": worker_node,
            "verifier": verifier_node,
        },
        flow=["worker", "verifier"],
        verify_node="verifier",    # this node triggers retry logic
        retry_from="worker",       # retry starts from this node
        max_retries=3,             # max 3 retries before moving on
    )
```

The `flow` list defines the node order. `verify_node` must be a node in `flow`.
After the verify node, the router decides: pass → next node in flow (or END), fail → `retry_from`.

## The `verification_result` State Convention

The verifier agent **must** write to `state["verification_result"]`. The retry router reads
`verification_result.get("status")` to make its decision.

```python
# Passing — route forward
{"status": "pass", "notes": "Output meets all requirements"}

# Failing — trigger retry
{"status": "fail", "error_message": "Missing required field: recommendations"}

# Also fails (status != "pass")
{"status": "retry", "error_message": "Sources are not cited"}
```

The `error_message` field is automatically appended to `retry_history` by `_make_retry_increment()`.

## Writing a Verifier Agent

Verifiers are typically pure reasoning agents — they read existing state and judge quality.
Use `generate()` instead of `run_react()` — no tools needed.

```python
class VerifierAgent(BaseAgent):
    def __init__(self):
        super().__init__(name="verifier", prompts_dir=PROMPTS_DIR)

    @property
    def system_prompt(self) -> str:
        return self.get_prompt_manager().assemble_prompt("verifier")

    @track_timing
    async def execute(self, state: dict) -> Command:
        result = state.get("result", {})
        task = state.get("tasks", [{}])[state.get("current_task_index", 0)]

        response = await self.generate(
            f"Task: {task.get('description', '')}\n\n"
            f"Output to verify:\n{result}\n\n"
            f"Check: does the output meet all requirements? "
            f"Return JSON with status (pass/fail) and error_message if failing."
        )
        data = self.extract_json(response) or {"status": "pass", "notes": response}
        return Command(goto="__end__", update={"verification_result": data})
```

Verifier prompt (`my_project/prompts/verifier.md`):

```markdown
# Verifier — Output Quality Checker

You review the worker agent's output and determine if it meets quality standards.
You do NOT use any tools. You only reason about what you are given.

## What to Check

1. All required fields are present in the output
2. Field values are non-empty and meaningful (not placeholder text)
3. The output addresses the original task
4. Any specific quality criteria for this pipeline

## Output Format

{"status": "pass", "notes": "Output meets all requirements."}

OR if failing:

{"status": "fail", "error_message": "Specific description of what is wrong and what the worker should fix."}

Output ONLY the JSON.
```

## How the Worker Reads `retry_history` for Self-Correction

On retry attempts, `retry_history` in state contains feedback from previous failures.
Read it in the worker to adjust behavior:

```python
@track_timing
async def execute(self, state: dict) -> Command:
    tasks = state.get("tasks", [])
    idx = state.get("current_task_index", 0)
    task = tasks[idx] if idx < len(tasks) else {}

    retry_history = state.get("retry_history", [])
    retry_count = state.get("retry_count", 0)

    # Build self-correction context
    correction_context = ""
    if retry_history:
        correction_context = (
            f"\n\nPREVIOUS ATTEMPT FEEDBACK (attempt {retry_count}):\n"
            + "\n".join(f"- {h.get('feedback', '')}" for h in retry_history)
            + "\n\nCorrect the issues listed above in your new output."
        )

    result = await self.run_react(
        task=(
            f"Complete this task:\n{task.get('description', '')}"
            + correction_context
        ),
        tools=get_research_tools(),
    )
    data = self.extract_json(result) or {"raw": result}
    return Command(goto="verifier", update={"result": data})
```

## Manual Retry Without the Factory

If you need more control (different retry logic per condition, custom routing), build the
retry loop manually using `add_conditional_edges`:

```python
from langgraph.graph import END, START, StateGraph

def create_pipeline():
    worker = WorkerAgent()
    verifier = VerifierAgent()

    async def worker_node(state): return await worker.execute(state)
    async def verifier_node(state): return await verifier.execute(state)

    async def increment_retry(state) -> dict:
        count = state.get("retry_count", 0)
        history = state.get("retry_history", [])
        verification = state.get("verification_result", {})
        history.append({"attempt": count + 1, "feedback": verification.get("error_message", "")})
        return {"retry_count": count + 1, "retry_history": history}

    def route_after_verify(state) -> str:
        verification = state.get("verification_result", {})
        retry_count = state.get("retry_count", 0)
        if verification.get("status") == "pass":
            return "continue"
        if retry_count < 3:
            return "retry"
        return "continue"  # exhausted retries

    graph = StateGraph(MyState)
    graph.add_node("worker", worker_node)
    graph.add_node("verifier", verifier_node)
    graph.add_node("increment_retry", increment_retry)

    graph.add_edge(START, "worker")
    graph.add_edge("worker", "verifier")
    graph.add_conditional_edges(
        "verifier",
        route_after_verify,
        {"continue": END, "retry": "increment_retry"},
    )
    graph.add_edge("increment_retry", "worker")

    return graph.compile()
```

## When to Use Retry vs Just Improving the Prompt

**Use retry when:**
- Output quality is objectively verifiable (schema completeness, data validity)
- Errors are recoverable with more information or a corrected attempt
- You need a quality gate for a downstream system

**Just improve the prompt when:**
- The agent consistently misunderstands the task
- Output format is wrong every time (fix the prompt's Output Format section)
- Agent loops (max_rounds hit) — this is a prompt/tool issue, not a retry issue

Retry adds latency and cost. Fix the underlying prompt first; add retry only when the
task is genuinely uncertain and requires multiple attempts.
