---
name: watcher-pipeline
title: Watcher Pipeline
description: Engine substrate for the inbox watcher → work queue → agent-mediated ingest dispatch chain. Tunes timeouts, heartbeat thresholds, and dedup. Never injected into any agent prompt.
category: _meta
status: active
author: system
metadata:
  priority: 0
---

# Watcher Pipeline (engine substrate)

This skill is **engine config**, not agent strategy. Its body is never injected into
agent system prompts. It exists so the dispatch pipeline's tuning lives in YAML
(Rule 12) instead of Python literals.

## Architecture (for future readers)

The inbox watcher's only job is **detect new files and enqueue ingest jobs to the
work queue**. It does NOT dispatch the agent inline. The work queue's `ingest`
handler is the SOLE place that calls `agent.invoke_skill("kb-ingest", ...)`.

This separation prevents the failure mode that the legacy inline-dispatch
architecture suffered from: a single stuck agent dispatch could hold the watcher
loop for up to 10 minutes (`_AGENT_INGEST_TIMEOUT_SECONDS=600`) blocking 108
consecutive cycles from doing any work. Decoupled, the watcher cycle returns in
milliseconds; the work queue handles dispatch + retry + escalation in the
background.

CI guard: [`tests/arch/test_watcher_no_inline_dispatch.py`](../../../tests/arch/test_watcher_no_inline_dispatch.py)
locks Phase 1's choice — any future call to `agent.invoke_skill` from inside
`automation/raw_watcher.py` fails the build.

## Knobs (config/dispatch.yaml)

- **`dispatch.timeout_seconds`** (default `600`) — Hard cap on a single
  `agent.invoke_skill("kb-ingest")` call inside the work-queue handler. Generous
  because local 27B models doing multi-step tool calls legitimately take 5-10
  minutes. Cloud is faster but the cap is one-size-fits-all.

- **`dispatch.long_dispatch_warning_seconds`** (default `60`) — Emit a medium-
  importance `ingest_agent_long_dispatch` activity event after this many seconds
  so the UI shows the user it's alive instead of looking stuck.

- **`enqueue.dedup_by_slug`** (default `true`) — When the watcher detects files
  in a KB inbox, it enqueues ONE job per slug regardless of file count. The work
  queue's `deduplicate=True` then collapses duplicate enqueues to the same KB
  while a job is pending. Re-enqueue when library evidence appears (next cycle
  detects ingest didn't complete) is the safety net.

## Why YAML, not constants

Engineers tuning this for a specific deployment (slow local hardware, expensive
cloud LLM, fast SSD) edit YAML and restart. No Python change, no PR review, no
arch-test churn. The fallback constants in code are tagged
`# arch:deterministic` and serve as the deg degraded-mode safety net when the YAML
is missing or malformed.
