# Context Management

Anteroom tracks token usage across the conversation and automatically manages context window limits.

## Token Tracking

Token counting uses the same request-shape estimator for CLI preflight checks, the context footer, and fallback usage when providers omit native usage. Native provider usage remains authoritative after a response. For estimates, Anteroom counts system prompts, dynamic system prompts, messages, tool schemas, assistant tool calls, tool results, structured content blocks, and response text.

The estimator uses **tiktoken** (`cl100k_base` encoding) when available, with a deterministic character fallback when it is unavailable. Structured text blocks are counted by their text fields instead of by a raw dictionary string. Dense JSON and tool schemas use a conservative guard so punctuation-heavy payloads are not clearly undercounted. Opaque image and document parts use bounded estimates rather than counting full data URLs or base64 payloads.

Warnings, footer counts, CLI preflight auto-compact, web warning metadata, and shared proactive compaction all compare thresholds against the active full-request estimate: message history, the base system prompt, any dynamic extra system prompt, and tool schemas.

| Threshold | What happens |
|---|---|
| **80,000 tokens** | Yellow warning and proactive historical tool-result collapse can run by default |
| **90,000 tokens** | Shared agent loop proactive summary compaction triggers by default |
| **100,000 tokens** | CLI request preflight auto-compacts before storing the next prompt |
| **128,000 tokens** | Effective context window ceiling |

## Context Footer

After each response, the context footer shows:

```
  [====----------------] 12,340/128,000 tokens (10%) | response: 482 | 3.2s | 87,660 until auto-compact
```

- **Progress bar** with color coding: green (<50%), yellow (50--75%), red (>75%)
- Current token count / max context window
- Response token count for the current turn
- Total elapsed thinking time
- Remaining tokens until the CLI preflight auto-compact threshold

## Compact

Use `/compact` to manually summarize and compact the conversation history.
Automatic compaction can also trigger without a slash command:

- The shared agent loop runs proactive summary compaction at
  `compaction.summary_trigger_token_count` (default `90,000`) or
  `compaction.summary_trigger_msg_count` (default `80`).
- Before summary compaction, the shared agent loop can proactively collapse
  older tool results at `compaction.historical_tool_collapse_trigger_token_count`
  (default `80,000`).
- The CLI request preflight still uses `cli.context_auto_compact_tokens`
  (default `100,000`) before it stores the pending prompt, using the same
  full-request estimate as the footer.

### Boundary-based compaction

By default, compaction preserves a **tail** of recent messages intact and only summarizes the older portion of the conversation. This keeps recent tool-use context, file edits, and conversational flow while still reducing token pressure.

The compacted shape looks like:

```
[summary message]      <- AI-generated summary of older messages
[user message]         <- first preserved tail message (always role: user)
[assistant message]
[tool results]
...                    <- remaining recent messages preserved intact
```

Three rules guarantee a safe split point:

1. **No orphaned tool results** — the split never lands mid-tool-sequence.
2. **Provider-safe ordering** — the preserved tail always starts with a `role: "user"` message, satisfying Anthropic and Bedrock validation.
3. **Fallback** — if no valid split exists (e.g. too few messages), the entire conversation is summarized as before.

Configure how many recent messages to preserve:

```yaml
compaction:
  preserve_tail: 6    # default; set to 0 for full-summary behavior
```

Or via environment variable: `AI_CHAT_COMPACT_PRESERVE_TAIL=6`.

The legacy `cli.compact_preserve_tail` key is still accepted as a deprecated alias.

### What the summary preserves

The AI generates a concise summary preserving:

- Key decisions and conclusions
- File paths that were read, written, or edited
- Important code changes and their purpose
- Current task state
- Errors encountered and how they were resolved

Tool call outputs are truncated to 500 characters each in the summary prompt. After compacting:

```
  Compacted 47 messages -> 7 messages
  ~32,140 -> ~4,890 tokens
```

!!! info
    Minimum 4 messages required to compact. This prevents compacting a nearly-empty conversation.

### Summary prompt recovery

For very large conversations, the summary request itself can exceed the model's
context window. When the provider returns a structured `context_length_exceeded`
error for the compaction summary, Anteroom retries within
`compaction.summary_retry_max_attempts` (default `3`). Each retry drops the
oldest safe turn groups from the summary input using
`compaction.summary_retry_drop_groups` (default `2`), while keeping assistant
tool calls attached to their tool results.

This recovery is intentionally lossy and only applies to prompt-too-long
summary failures. Generic provider failures do not trigger lossy drops. If
recovery exhausts its attempts, the original in-memory and persisted
conversation history is left unchanged.

Bulky image and document content is omitted from the summary request and
replaced with markers such as `[image omitted from compaction summary]` and
`[document omitted from compaction summary]`. The marker tells the summarizer
that the content existed without sending base64 media or document payloads.

```yaml
compaction:
  summary_max_completion_tokens: 4096
  summary_retry_max_attempts: 3
  summary_retry_drop_groups: 2
```

### Session state rehydration

In addition to the LLM-generated summary, the compaction service appends a
**bounded, deterministic** `<session_state>` block to the summary message
content. It captures structured context extracted directly from tool call
arguments and results — never LLM-generated — so the next turn retains a
reliable pointer to recent working state:

```
<session_state>
Files read: src/foo.py, src/bar.py, tests/test_foo.py
Files written: src/baz.py
Files edited: src/foo.py
Working dir: /path/to/repo
Last errors: bash: ModuleNotFoundError: No module named 'xyz'
</session_state>
```

Only non-empty subsections are included. If the summarised region has no
tool calls at all, no block is appended.

**What goes in:**

- File paths touched by `read_file`, `write_file`, `edit_file` (deduplicated,
  capped, most recent kept per category)
- Last working directory hinted by `bash` `cd <path>` segments or an explicit
  `cwd` kwarg
- Unresolved tool errors — an error is considered resolved if a later call
  with the same retry identity succeeds. File tools key on path; bash keys
  on the leading command token (e.g. `pytest`, `python:pytest` for
  `python -m pytest`) so a later unrelated bash success (`ls`) does not
  silently resolve an earlier failed `pytest`

**What does NOT go in:** plan state, RAG sources, system prompt, space
instructions, or skill catalog — these survive compaction already because
they live outside the message list.

Configure via:

```yaml
compaction:
  compact_rehydrate: true              # default; set false to disable
  compact_rehydrate_max_files: 20      # per-category cap (read/written/edited)
  compact_rehydrate_max_errors: 5      # max unresolved errors preserved
```

Or via environment variables: `AI_CHAT_COMPACT_REHYDRATE`,
`AI_CHAT_COMPACT_REHYDRATE_MAX_FILES`, `AI_CHAT_COMPACT_REHYDRATE_MAX_ERRORS`.

On re-compaction, any prior `<session_state>` block is stripped from the
compaction input before summarisation and from the LLM output before the
fresh block is appended — so the block is never nested or duplicated.

## Proactive summary compaction

Before each LLM call, the agent loop estimates the active full request
and runs summary compaction when either configured threshold is reached:

- `compaction.summary_trigger_token_count` (default `90000`)
- `compaction.summary_trigger_msg_count` (default `80`)

The token threshold includes fixed system/tool overhead as well as messages.
That means a large system prompt, dynamic context injection, or large tool
schema set can trigger warnings or compaction even when the visible message
history is short.

The compacting status includes the reason and counts, for example
`token threshold 128,500/128,000`, `message threshold 84/80`, or
`context-error recovery`. This distinction matters: seeing a compacting
status while token usage is below your configured token threshold usually
means the message-count threshold fired, not that the token threshold was
ignored.

Proactive compaction is reported as a lifecycle/status phase rather than as
assistant text. It should not be saved as a model response in the transcript.
If the conversation already begins with a compact summary, Anteroom does not
immediately compact it again just because the retained compacted prefix or
preserved tail still makes the total request look large. It waits until new
post-compaction history crosses the token or message threshold. This also
prevents repeated no-op compactions when the remaining pressure comes mostly
from fixed system/tool overhead that summarising messages cannot reduce.

## Proactive microcompact

Before each LLM call, the agent loop runs a cheap, deterministic,
**in-memory** history sweep that trims oversized content before it can
trigger a context overflow. Microcompact is complementary to the
per-tool-ingestion compaction (`services/tool_result_compact.py`) and the
reactive overflow ladder below — it runs *proactively* every turn.

Microcompact ships two strategies:

| Strategy | Effect |
|---|---|
| **`oversized_tool_outputs`** | Sweeps every `role: "tool"` message whose content exceeds `cli.tool_output_max_chars` and replaces it with the truncated head plus a retry hint. Same strategy implementation as reactive Strategy 1 (single source of truth). |
| **`historical_tool_results`** | When estimated history tokens reach `compaction.historical_tool_collapse_trigger_token_count`, compacts older tool results to `compaction.historical_tool_collapse_compact_chars` while preserving the most recent `compaction.historical_tool_collapse_keep_recent_groups` turn groups. It preserves structured fields such as `status`, `path`, `exit_code`, and `error` when they fit. |

Microcompact is **in-memory only** — it never writes to SQLite. The
trade-off is that trims are not durable across session resume; a resumed
conversation starts fresh, and microcompact will sweep again on the next
turn if oversized or historical tool outputs still meet the configured
thresholds in the loaded history. It mutates only the message list sent to
the model for the current loop.

When at least one message changes, the agent loop emits a short notification
token. Oversized trimming prints `*Trimmed oversized tool outputs to stay
within context limits...*`. Historical collapse prints `*Collapsed older tool
results to reduce context pressure...*` and emits compaction metadata with
the strategy name, bytes saved, modified count, preserved group count, compact
character budget, token estimate, and trigger threshold. CLI renders the
notice inline; web streaming receives the same phase/token data and can render
or ignore it safely.

Disable the stage entirely with:

```yaml
compaction:
  microcompact_enabled: false
```

Or via environment variable: `AI_CHAT_MICROCOMPACT_ENABLED=false`.

Configure historical collapse with:

```yaml
compaction:
  historical_tool_collapse_enabled: true
  historical_tool_collapse_trigger_token_count: 80000
  historical_tool_collapse_keep_recent_groups: 6
  historical_tool_collapse_compact_chars: 1000
```

Or via environment variables:
`AI_CHAT_HISTORICAL_TOOL_COLLAPSE_ENABLED`,
`AI_CHAT_HISTORICAL_TOOL_COLLAPSE_TRIGGER_TOKEN_COUNT`,
`AI_CHAT_HISTORICAL_TOOL_COLLAPSE_KEEP_RECENT_GROUPS`, and
`AI_CHAT_HISTORICAL_TOOL_COLLAPSE_COMPACT_CHARS`.

This proactive historical collapse is different from reactive overflow
recovery. Proactive collapse runs before the model call only when token
pressure reaches the configured threshold; reactive recovery runs only after
the provider returns `context_length_exceeded`. It is also different from full
summary compaction: no LLM call is made, no summary message is created, and no
collapsed historical tool result is persisted.

## Staged reactive overflow recovery

When a request to the model actually exceeds the context window and the provider
returns a `context_length_exceeded` error, the agent loop recovers through four
staged strategies in order. Cheaper strategies fire first so the conversation is
preserved as much as possible — full LLM compaction is only used as a last resort.

| # | Strategy | Cost | Effect |
|---|----------|------|--------|
| 1 | **Truncate oversized tool outputs** | Free (byte slicing) | Trims any individual `role: tool` message whose content exceeds `tool_output_max_chars`, appending a retry hint |
| 2 | **Collapse historical tool results** | Free (structured truncation) | Compacts *all* tool-result messages in older turn groups to ~200 chars, preserving structured fields (status, exit_code, error, path) |
| 3 | **Drop older turn groups** | Free (deterministic summary) | Drops older turn groups and replaces them with a compaction summary message matching `/compact` output shape. Preserves the most recent 4 turn groups |
| 4 | **Full LLM compaction** | One LLM call | Summarises the whole older portion via the model. Same behaviour as `/compact` |

Each strategy reports whether it made progress. The first one that reports progress
wins and the loop retries the request. If none make progress, the loop surfaces a
recovery-failed error.

All intermediate strategies are **turn-group aware** — they never split an
`assistant` message with `tool_calls` from its subsequent `role: "tool"` results.
Strategy 3 delegates boundary selection to the same walk-back rules used by
`compact_messages()` so the preserved tail is always provider-safe (starts with
`role: "user"`).

Each strategy emits a short notification token stream visible in the UI:

- `*Context limit reached — tool output was too large. Truncated and retrying with smaller scope...*`
- `*Context limit reached — collapsed historical tool results, retrying...*`
- `*Context limit reached — dropped older conversation turns, retrying...*`
- `*Context limit reached — compacting conversation and retrying...*`
- `*Compaction summary prompt was too long — dropped older safe turns and retried...*`

When Strategy 4 runs, the lifecycle status also identifies the reason as
`context-error recovery`, which is separate from proactive token or
message-count compaction.

The retry cap (default 4) is configurable via
`compaction.reactive_max_attempts` (env: `AI_CHAT_REACTIVE_MAX_ATTEMPTS`).
Strategy 4's LLM summary is cancel-aware — pressing Escape (CLI) or
disconnecting (web) aborts the provider call cleanly instead of hanging.
If Strategy 4 cannot recover from a prompt-too-long summary request, it emits a
terminal error and leaves the original history unchanged.
