Metadata-Version: 2.4
Name: bottrace
Version: 0.1.0
Summary: A headless CLI debugging controller for Python
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Debuggers
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# bottrace

A headless CLI debugging controller for Python. Designed for both humans and LLM agents.

## Installation

```bash
pip install -e .
```

## Quick Start

```bash
# Basic call tracing
bottrace run app.py --calls

# See what functions are called most
bottrace run app.py --call-counts

# Debug an exception
bottrace run buggy.py --on-exception snapshot --locals

# Breakpoint snapshots (capture state at specific lines)
bottrace run app.py --breakpoint app/core.py:42 --watch 'len(items)' --json-out trace.json

# Bounded execution (prevent runaway traces)
bottrace run app.py --calls --timeout 10 --max-events 1000
```

## Core Concept

**bottrace is not a debugger. It's a debugging controller.**

It wraps Python's `sys.settrace()` and related primitives to emit structured, machine-parseable output. This lets you understand runtime behavior without setting breakpoints or adding print statements.

**Key principle:** Runtime execution is authoritative. Symbol names and static analysis are secondary to observed call paths.

## Commands

### `bottrace run`

Execute a Python script with optional tracing and debugging controls.

```
bottrace run <script.py> [script_args...] [options]
```

## Options

### Tracing Options

| Flag | Description |
|------|-------------|
| `--calls` | Emit CALL/RETURN events for each function |
| `--call-counts` | Show aggregated call frequency summary (Note: Misses C → Python calls) |
| `--no-comprehensions` | Suppress comprehension/genexpr call events |
| `--max-depth N` | Limit trace depth to N levels |

### Filtering Options

| Flag | Description |
|------|-------------|
| `--include PREFIX` | Only trace files under this path (repeatable) |
| `--exclude PREFIX` | Exclude files under this path (repeatable) |

By default, bottrace traces only your project code (script's directory), not site-packages or stdlib.

### Exception Options

| Flag | Description |
|------|-------------|
| `--on-exception snapshot` | On uncaught exception, dump stack trace |
| `--locals` | Include local variables in exception snapshot |

### Bounding Options

| Flag | Description |
|------|-------------|
| `--timeout N` | Kill execution after N seconds (exit code 124) |
| `--max-events N` | Stop tracing after N events (program continues) |

### Breakpoint Snapshot Options

| Flag | Description |
|------|-------------|
| `--breakpoint FILE:LINE` | Capture snapshot when execution reaches file:line (repeatable) |
| `--watch EXPR` | Expression to evaluate at breakpoints (repeatable) |
| `--snapshots N` | Stop capturing after N snapshots |
| `--json-out PATH` | Write snapshots to JSON file |
| `--verbose` | Print snapshot summaries to stderr (useful without JSON) |
| `--print-result` | Print JSON result summary to stdout (for agent discovery with --json-out) |

Breakpoint snapshots are non-interactive - execution auto-continues after each capture. Designed for CI/CD and LLM agent workflows.

**Important: Breakpoint Placement**

Set breakpoints on lines **inside** function bodies, not on `def` lines:

```python
def process(item):      # Line 10 - DON'T set breakpoint here (captures at import time)
    result = item * 2   # Line 11 - SET breakpoint here (captures at call time)
    return result       # Line 12 - Or here
```

The `def` line executes when the module is imported (to create the function object), not when the function is called. To capture state during function execution, use a line inside the function body.

## Output Format

### Call Trace

```
CALL main app.py:10
CALL load_config app.py:5
RETURN load_config
CALL process app.py:15
RETURN process
RETURN main
```

### Call Counts

```
Call Counts (Top 10):
 100 calls to inner (worker.py:12)
   1 calls to main (app.py:5)
   1 calls to outer (app.py:10)
```

### Exception Snapshot

```
Exception: KeyError(user_id)

Stack:
  get_user() db.py:42
    Locals:
      user_id: 'abc123'
      conn: <Connection connected=True>
  handle_request() api.py:28
  main() app.py:10
```

### Timeout

```
TIMEOUT after 5s - dumping stacks:
Thread MainThread:
  infinite_loop() slow.py:7
  main() slow.py:2
```

### Breakpoint Snapshot (JSON)

```json
{
  "version": 1,
  "snapshots": [
    {
      "breakpoint": "app/core.py:42",
      "timestamp": 1706640000.123,
      "stack": [
        {
          "function": "process_item",
          "filename": "/home/user/app/core.py",
          "lineno": 42,
          "locals": {"item": "{'id': 123}", "count": "5"}
        },
        {
          "function": "main",
          "filename": "/home/user/app/main.py",
          "lineno": 25,
          "locals": null
        }
      ],
      "watches": [
        {"expression": "len(items)", "value": "42", "error": null},
        {"expression": "self.state", "value": "'running'", "error": null}
      ]
    }
  ]
}
```

## Exit Codes

| Code | Meaning |
|------|---------|
| 0 | Success |
| 1-123 | Script's exit code (passed through) |
| 124 | Timeout |

## Usage Patterns for Agents

### 1. Find all code executed by a feature

```bash
# Trace the feature
bottrace run manage.py migrate --calls > trace.txt

# Extract unique files touched
grep "^CALL" trace.txt | awk '{print $3}' | cut -d: -f1 | sort -u

# Get call frequency
bottrace run manage.py migrate --call-counts
```

### 2. Disambiguate same-named symbols

When multiple modules define `FooHandler`, prove which one runs:

```bash
bottrace run app.py handle_request --calls | grep FooHandler
# Output: CALL FooHandler handlers/v2/foo.py:42
```

### 3. Verify no forbidden dependencies

Confirm `NewClass` doesn't call into legacy code:

```bash
bottrace run test_new_class.py --calls > trace.txt
grep "legacy_module" trace.txt
# Empty = pass, any output = violation
```

### 4. Debug without adding logging

```bash
# Use existing logging + bottrace together
LOG_LEVEL=DEBUG bottrace run app.py --calls --max-depth 5 2>&1 | tee output.txt
```

### 5. Find why something is called 10,000 times

```bash
bottrace run app.py --call-counts
# Shows: 10000 calls to validate_item (validator.py:12)
```

### 6. Understand end-to-end flow

```bash
bottrace run cli.py deploy --calls --max-depth 4
# Shows: boot -> dispatch -> business logic -> side effect
```

### 7. Debug an exception

```bash
bottrace run buggy.py --on-exception snapshot --locals
# Shows stack + local variables at crash point
```

### 8. Capture state at specific locations (breakpoint snapshots)

```bash
# Capture stack + locals when hitting specific lines
bottrace run app.py \
  --breakpoint app/core.py:42 \
  --breakpoint app/api.py:88 \
  --watch 'len(items)' \
  --watch 'request.user_id' \
  --snapshots 5 \
  --json-out debug.json

# Then analyze the JSON
jq '.snapshots[0].stack[0].locals' debug.json
jq '.snapshots[] | .watches[] | select(.error == null)' debug.json
```

This is ideal for:
- Capturing state in CI/CD without interactive debugging
- LLM agents that need structured debug data
- Repeatable "debug snapshots" for regression analysis

## LLM Agent Workflow

When using bottrace as an LLM agent, follow this 3-step process:

### Step 1: Get the Shape

Start with `--call-counts` to understand what's happening without drowning in detail:

```bash
bottrace run app.py --call-counts --timeout 5
```

Output shows the most-called functions with exact file:line locations:
```
Call Counts (Top 10):
 892 calls to __init__ (unified_atlas_manager.py:144)
 101 calls to _ (i18n.py:133)
  88 calls to __init__ (sound_manager.py:41)
```

### Step 2: Map Files Touched

Extract unique project files executed:

```bash
bottrace run app.py --calls --max-events 5000 2>&1 | \
  grep "^CALL" | awk '{print $3}' | cut -d: -f1 | sort -u
```

This gives you the complete list of files involved in the execution path.

### Step 3: Drill Into Specific Areas

Focus on a specific module or limit depth:

```bash
# Focus on one module
bottrace run app.py --calls --include src/auth/ --max-events 1000

# Or limit call depth for high-level view
bottrace run app.py --calls --max-depth 3 --max-events 1000
```

## Real-World Example: Tracing a Pygame Game

Tested on a 199-file Pygame/OpenGL game:

```bash
# Run headlessly with virtual framebuffer
SDL_AUDIODRIVER=dummy xvfb-run -a \
  bottrace run src/main.py --windowed --call-counts --timeout 5
```

### Finding Hotspots

```bash
bottrace run src/main.py --calls --max-events 50000 2>&1 | \
  grep "^CALL" | awk '{print $2, $3}' | sort | uniq -c | sort -rn | head -10
```

Output:
```
10091 <genexpr> zaxxon_turret.py:766
 3127 <genexpr> zaxxon_turret.py:814
 1541 normalize_vec3 math_helpers.py:456
  786 add_vec3 math_helpers.py:475
```

This immediately reveals that `zaxxon_turret.py` has hot generator expressions worth investigating.

### Symbol Disambiguation

When multiple modules define the same function (e.g., `normalize`):

```bash
grep -rn "def normalize" src/
# src/math_helpers.py:456:def normalize_vec3(v)
# src/behaviors/super_formation_join_behavior.py:361:def normalize(angle)
# src/bullet.py:109:def normalize(self, vector)
```

bottrace proves which one actually runs:
```
1541 calls to normalize_vec3 (math_helpers.py:456)
```

### Decoupling Verification

Verify that module A doesn't call into forbidden module B:

```bash
# Capture full trace
bottrace run app.py --calls --max-events 50000 > trace.txt

# Check: does camera.py call adversary code?
grep "camera.py" trace.txt | grep -c "adversary"
# Output: 0 (PASS - decoupled)

# Check: does i18n call rendering code?
grep "i18n.py" trace.txt | grep -c "renderer"
# Output: 0 (PASS - decoupled)
```

### High-Level Flow with --max-depth

```bash
bottrace run src/main.py --calls --max-depth 2 --max-events 500
```

Output shows boot sequence without internal details:
```
CALL <module> src/main.py:1
CALL <module> src/TwoD.py:1
RETURN <module>
CALL <module> src/game_state.py:1
RETURN <module>
...
CALL main src/main.py:1447
RETURN main
```

## Progressive Narrowing Workflow

bottrace is designed for iterative refinement:

1. **Broad trace** - `bottrace run app.py --calls`
2. **Identify noise** - Note irrelevant modules
3. **Narrow** - `bottrace run app.py --calls --exclude tests/ --max-depth 3`
4. **Repeat** - Until you see only relevant code

## Combining Flags

Flags can be placed before or after the script path:

```bash
# Both work:
bottrace run --calls --max-depth 3 app.py
bottrace run app.py --calls --max-depth 3

# Script args come last:
bottrace run app.py --calls -- --script-arg value
```

## Examples

The `examples/` directory contains test scripts:

- `hello.py` - Basic script with args
- `calls.py` - Nested function calls
- `exception.py` - Script that raises
- `slow.py` - Long-running script (for timeout testing)
- `many_calls.py` - High call volume (for call-counts testing)

## Limitations

- **C-Extension Boundary**: `sys.settrace()` does not fire `CALL` events when a C-extension calls a Python function (e.g. `pygame` main loop, `numpy` callbacks). These calls are missed by `--calls` and `--call-counts`. Workaround: use `--breakpoint` on the first line of the Python function.
- **No attach**: Cannot attach to running processes
- **No continuous line tracing**: Only call/return events by default (use `--breakpoint` for specific lines)
- **No async tracing**: Does not trace async task switches
- **Watch expressions**: Limited to safe builtins (no imports, no side effects)

## Design Philosophy

1. **CLI-first** - No UI, works over SSH/CI/containers
2. **Unix composable** - Pipe to grep, jq, awk
3. **Agent-friendly** - Deterministic output, explicit flags
4. **Bounded by default** - Always use caps to prevent drowning
5. **Execution is truth** - Runtime traces beat static analysis

## stdout/stderr Contract

**Golden rule: STDOUT is data. STDERR is commentary.**

This enables clean Unix piping and agent-friendly parsing.

### What Goes Where

| Output Type | Stream | Example |
|-------------|--------|---------|
| CALL/RETURN trace events | stdout | `CALL main app.py:10` |
| Call counts summary | stdout | `100 calls to inner (worker.py:12)` |
| NDJSON snapshots (no --json-out) | stdout | `{"breakpoint":...}` |
| Exception snapshots | stdout | `Exception: KeyError(...)` |
| `--print-result` JSON | stdout | `{"result_file":"...","snapshots":3}` |
| Errors and warnings | stderr | `bottrace: error: script not found` |
| `--verbose` summaries | stderr | `SNAPSHOT #1 at file:42` |
| MAX_EVENTS/MAX_SNAPSHOTS notices | stderr | `MAX_EVENTS reached (5)` |
| Timeout messages | stderr | `TIMEOUT after 5s - dumping stacks` |

### Agent Discovery with `--print-result`

When writing to a JSON file, use `--print-result` for deterministic artifact discovery:

```bash
bottrace run app.py --breakpoint app.py:42 --json-out trace.json --print-result
# STDOUT: {"result_file": "trace.json", "snapshots": 3}
```

This lets agents capture the result without parsing stderr:

```bash
RESULT=$(bottrace run ... --json-out trace.json --print-result)
FILE=$(echo "$RESULT" | jq -r '.result_file')
COUNT=$(echo "$RESULT" | jq -r '.snapshots')
```

## For Agent Developers

When training an agent to use bottrace:

1. **Start with `--call-counts`** - Get the shape before drilling down
2. **Always bound output** with `--max-events` (5000-50000) or `--timeout`
3. **Use `--max-depth 2-3`** for architecture understanding
4. **Extract file lists** with grep/awk pipeline for codebase mapping
5. **Verify decoupling** by grepping traces for forbidden modules
6. **Cite evidence** - conclusions should reference bottrace output with file:line
7. **Don't add logging** - use `--on-exception snapshot --locals` instead

### Output Parsing Tips

The output format is designed for Unix pipelines:

```bash
# Count calls per file
grep "^CALL" trace.txt | awk '{print $3}' | cut -d: -f1 | sort | uniq -c | sort -rn

# Find all calls to a specific function
grep "^CALL specific_func" trace.txt

# Check if module X calls module Y
grep "module_x" trace.txt | grep -c "module_y"  # 0 = no coupling
```

### Recommended Bounds

| Scenario | Recommended Flags |
|----------|-------------------|
| Quick overview | `--call-counts --timeout 5` |
| File mapping | `--calls --max-events 5000` |
| Architecture | `--calls --max-depth 3 --max-events 1000` |
| Focused debugging | `--calls --include src/module/ --max-events 2000` |
| Exception analysis | `--on-exception snapshot --locals` |
| Breakpoint debugging | `--breakpoint file:line --snapshots 5 --json-out trace.json` |
| Quick breakpoint check | `--breakpoint file:line --verbose` (no JSON, summaries to stderr) |

See `docs/USE_CASES.md` for detailed agent workflow patterns.
