---
name: reply-guardrails
title: "Reply Guardrails"
description: "Engine substrate for detecting placeholder fabrication and hallucinated capability refusals in agent chat replies"
category: _meta
status: active
priority: 9
---

## Instructions

This is a `_meta` skill — an engine-level substrate, not a behavioral skill. It is not injected into any agent's system prompt. Its purpose is to document the reply-guardrails subsystem and expose its tuning knobs as YAML so strategy lives outside Python (Rule 12).

The engine that reads this config is [automation/reply_guardrails.py](../../../../server/src/agntspace/automation/reply_guardrails.py). Two handlers register against the `before_agent_reply` hook on every chat turn:

### 1. Placeholder scrub handler

Runs first. Scans the reply for bracketed placeholder tokens the LLM emits when it couldn't ground a field (`[Your Name]`, `[Subject]`, `[Topic]`). Each detected placeholder is replaced with `⟨missing⟩`. If the reply contains more than `max_placeholders_before_refusal` placeholders, the entire reply is replaced with a refusal message — that many missing fields means the agent was fabricating at scale, not fumbling one detail.

Every scrub emits a high-importance `fabrication_in_chat` activity event. The user sees the scrub in the reply AND the system can audit it later.

### 2. Capability-hallucination handler

Runs second. Scans the reply for refusal phrases ("I'm currently unable to", "I can't create", "you'll have to do it manually"). If a refusal phrase matches AND a capability keyword within `context_window_chars` of it maps to a tool the agent actually has in the registry, the refusal is a hallucination.

The handler does NOT mutate the reply — the user still reads what the agent said — but it writes a decision log record with:
- `action_type = "reply_signal"`
- `contract_result = "capability_hallucinated"`
- `rationale` = the refusal sentence + the matched capability + the available tool

SkillSchool reads these records every heartbeat cycle and proposes skill tunings (same pattern as `zero_effect` decisions). Repeated capability hallucinations on the same capability → propose a skill that teaches the flow, or tune an existing one to add tools_granted.

## The load-bearing rules

1. **Scrub ≠ gate.** Placeholder scrub is a last-line safety net; the primary defense is anti-fabrication rules in the skill that governs the task (see `email-compose`, `memory-consolidate`, etc.). If the scrub is firing often, the fix is the upstream skill, not the scrub threshold.

2. **Hallucination detection does not mutate replies.** The agent's reply is its own output. The handler only logs the signal. Gating replies on capability would produce a different, worse failure mode (vetoed replies on edge cases).

3. **Both handlers fail open.** If the engine module, the YAML, or the activity/decision loggers are missing, the chain continues unchanged. The handlers never veto. A broken guardrail must never kill a chat turn.

4. **No hardcoded phrase lists.** Refusal phrases, placeholder patterns, capability keywords — all live in `config/guardrails.yaml`. Adding a new language means editing YAML, not Python.

## Tuning

Edit `config/guardrails.yaml`:

- `placeholders.patterns` — add regex for new placeholder shapes the LLM invents
- `placeholders.ignore_patterns` — whitelist structural uses of brackets (wikilinks, markdown, code)
- `placeholders.max_placeholders_before_refusal` — lower to be stricter, 0 to disable full-reply refusal
- `capabilities.refusal_phrases.<lang>` — add refusal phrases in new languages
- `capabilities.capability_keywords.<tool_name>` — teach the engine which natural-language verbs map to which tool
- `capabilities.enabled: false` / `placeholders.enabled: false` — disable one handler independently

## What this substrate does NOT do

- It does not teach the agent to avoid fabrication — that's the job of upstream skills.
- It does not solve "the agent hallucinated a tool" (the engine catches that at tool execution time, not at reply time).
- It does not gate the reply on contract — contract checking happens before the tool executes, long before the reply.
- It does not evaluate the quality of the reply beyond these two mechanical checks. Quality review is `critique-protocol`.
