# Phase 0 Integrated Implementation Plan

**Author:** Benjamin Booth  •  **Date:** 2026-04-13  •  **Status:** Active  •  **Branch:** `feat/phase-0-classroom-data-platform`

**Scope:** Classroom foundation + data platform + federation primitives + eval-driven validation, delivered as an integrated whole rather than sequential module builds.

**Why integrated:** Classroom alone is just another chat UI. Data platform alone is infrastructure nobody uses. Federation alone is plumbing with nothing to carry. The three only prove out when they compose — chat produces traces, traces flow through medallion layers, findings promote into RAG, federation replicates the useful ones, evals improve, research loops converge faster.

---

## Guiding Principles

1. **TDD throughout** — write the test first, watch it fail, implement the minimum to pass. No speculative code.
2. **Vertical slices, not horizontal layers** — each increment is green-and-pushable, proves an end-to-end behavior, and unlocks the next slice.
3. **Real infrastructure where possible** — avoid mock-heavy tests that pass without proving the system works. Reserve mocks for third-party APIs outside our control.
4. **Eval-driven** — every slice ends with a test that measures whether the system does what it should, not just whether the code compiles.
5. **Defer non-blocking decisions** — capture them in ADRs or follow-on tasks, but don't let them block Phase 0.

---

## Component Inventory

| Component | Category | ADR | Profile |
|-----------|----------|-----|---------|
| Trace provider protocol + Langfuse + noop | Observability | ADR-019 | All |
| OpenAI-compatible endpoint in `neut serve` | Chat infra | — | Edge+ |
| Open WebUI deployment on Rascal | Chat UI | — | Server+ |
| Eval harness + core scorers | Quality | — | All |
| Research loop engine (CURI-O core) | Intelligence | architecture/curio-research-loop.md | All |
| SeaweedFS | Storage | ADR-019 | Workstation+ |
| DuckDB integration | Query | ADR-019 | Workstation+ |
| Apache Iceberg | Table format | ADR-019 | Server+ |
| Dagster | Orchestration | ADR-019 | Server+ |
| Bronze/Silver/Gold medallion flow | Data | — | Workstation+ |
| RAG promotion from gold | Knowledge | prd-classroom §5.9.5 | All |
| Ed25519 identity keys (platform + node + context) | Security | ADR-020, ADR-021 | All |
| Signed federation messages (3-layer chain) | Federation | ADR-020, ADR-021 | All |
| Research digest exchange | Federation | curio-research-loop §9 | All |
| Per-peer rate limits + quarantine | Security | ADR-021 | All |
| Local reputation primitives | Federation | ADR-021 | All |

---

## Dependency Graph

```mermaid
flowchart TB
    TP["Trace provider + Langfuse"]
    EV["Eval harness + scorers"]
    OE["OpenAI-compat endpoint"]
    OW["Open WebUI on Rascal"]
    RL["Research loop engine"]
    SW["SeaweedFS"]
    DU["DuckDB"]
    MB["Medallion bronze"]
    MS["Medallion silver"]
    MG["Medallion gold → RAG promotion"]
    IC["Iceberg + Dagster"]
    KE["Identity keys + signed messages"]
    FE["Federation digest exchange"]
    FT["Two-node federation test"]
    UE["Eval-uplift proof"]

    TP --> EV
    TP --> OE
    TP --> RL
    OE --> OW
    EV --> RL
    SW --> MB
    DU --> MB
    TP --> MB
    MB --> MS
    MS --> MG
    MG --> IC
    RL --> FE
    KE --> FE
    MG --> FE
    FE --> FT
    EV --> UE
    FT --> UE

    style TP fill:#2b6cb0,color:#fff
    style EV fill:#2b6cb0,color:#fff
    style OE fill:#2b6cb0,color:#fff
    style OW fill:#2b6cb0,color:#fff
    style RL fill:#c05621,color:#fff
    style SW fill:#2f855a,color:#fff
    style DU fill:#2f855a,color:#fff
    style MB fill:#2f855a,color:#fff
    style MS fill:#2f855a,color:#fff
    style MG fill:#2f855a,color:#fff
    style IC fill:#2f855a,color:#fff
    style KE fill:#6b46c1,color:#fff
    style FE fill:#6b46c1,color:#fff
    style FT fill:#c05621,color:#fff
    style UE fill:#c05621,color:#fff
```

---

## Vertical Slices

Each slice ends with a specific, demoable behavior and passing tests. Estimate is focused-work days, not calendar.

### Slice 1: Hello Trace (~1d)

**Behavior:** `neut chat` emits a Langfuse trace for every turn. Noop fallback when Langfuse is not configured.

**Scope:**
- `axiom/src/axiom/infra/tracing/` with `provider.py`, `langfuse_provider.py`, `noop_provider.py`, factory
- Instrument `ChatAgent.turn()` with trace provider calls
- Config loader for `runtime/config/tracing.toml`

**Tests (TDD order):**
1. `test_trace_provider_protocol_exists` — the protocol type is importable and has expected methods
2. `test_noop_provider_accepts_all_calls` — noop swallows every call without error
3. `test_factory_returns_noop_when_no_config` — absent config → noop
4. `test_factory_returns_langfuse_when_configured` — config present → Langfuse instance
5. `test_chat_agent_emits_start_trace` — integration test against real Langfuse instance (or container)
6. `test_chat_agent_emits_log_generation_with_token_counts` — post-turn, trace has prompt_tokens and completion_tokens
7. `test_chat_agent_emits_log_retrieval_when_rag_used` — RAG retrieval produces a span

**Done:** `neut chat` runs against Langfuse running locally (Docker Compose), traces appear in Langfuse UI.

### Slice 2: Hello Evals (~1-2d)

**Behavior:** `axi eval run --suite hello.yaml --target "neut serve"` executes a 3-case eval suite and reports pass/fail with scores.

**Scope:**
- `axiom/src/axiom/extensions/builtins/evals/` extension
- `harness.py` — runner with parallelism
- `scorer.py` — exact_match, semantic_similarity, llm_judge scorers
- `suite.py` — YAML schema + loader
- `reporter.py` — terminal + JSON output
- CLI: `axi eval create-suite`, `axi eval run`, `axi eval report`

**Tests:**
1. `test_suite_yaml_schema_valid` — sample YAML parses to SuiteSpec
2. `test_exact_match_scorer_hit_miss` — string equality with normalization
3. `test_semantic_similarity_scorer_threshold` — embedding cosine ≥ threshold
4. `test_llm_judge_scorer_returns_numeric` — judge LLM produces a score via real LLM call
5. `test_harness_runs_all_cases` — N cases → N results
6. `test_harness_parallel_execution_faster_than_serial` — timing test
7. `test_report_includes_per_case_and_aggregate` — report has both
8. `test_eval_run_against_neut_serve_endpoint` — integration: eval against real running neut serve

**Done:** A `hello.yaml` eval suite runs against neut serve, reports correctly, exits 0/1 based on pass rate.

### Slice 3: Hello WebUI (~1-2d, partly deploy work)

**Behavior:** Open WebUI running on Rascal talks to Axiom backend via `/v1/chat/completions`. Student sees chat response with citations rendered.

**Scope:**
- `axiom/src/axiom/infra/openai_endpoint/` — OpenAI-compat endpoint in `neut serve`
- Streaming support (SSE)
- Open WebUI Docker Compose config added to Rascal deployment
- Caddy route for Open WebUI alongside Axiom API
- Config for Open WebUI to register Axiom as OpenAI-compatible model provider

**Tests:**
1. `test_openai_endpoint_chat_completions_non_streaming` — POST to `/v1/chat/completions` returns valid OpenAI response shape
2. `test_openai_endpoint_chat_completions_streaming` — SSE chunks emitted in correct format
3. `test_openai_endpoint_rejects_invalid_model` — unknown model → 404
4. `test_openai_endpoint_propagates_rag_retrieval_to_response` — response includes citation metadata
5. `test_openai_endpoint_emits_traces` — every call produces trace
6. **Manual acceptance:** Soha test — use Open WebUI on Rascal for 30 minutes, no crashes, citations visible

**Done:** Open WebUI accessible at `https://rascal.austin.utexas.edu/chat/`, Axiom backend serves it, Soha uses it for an hour and it doesn't fall over.

### Slice 4: Hello Research Loop (~2-3d)

**Behavior:** User types `/research "<topic>"`, CURI-O runs 2-4 iterations and converges on a trivial test case with a synthesized output. Loop state is serializable and resumable.

**Scope:**
- `axiom/src/axiom/agents/curio/loop_engine.py` — ResearchLoopEngine
- `state.py` — ResearchLoopState, SubQuestion, IterationLog, ConvergenceState (per curio-research-loop.md §7)
- `phases/formulate.py`, `search.py`, `evaluate.py`, `refine.py`, `promote.py` — each phase
- `convergence.py` — should_converge() logic
- `persistence.py` — serialize/deserialize loop state to disk
- CLI: `axi research start`, `status`, `resume`, `stop`, `list`, `show`

**Tests:**
1. `test_research_loop_state_round_trip_serialization`
2. `test_formulate_phase_produces_hypothesis_with_sub_questions`
3. `test_search_phase_returns_evidence_with_provenance`
4. `test_evaluate_phase_produces_support_score_and_contradictions`
5. `test_refine_phase_updates_hypothesis_status`
6. `test_convergence_triggers_on_novelty_decay`
7. `test_convergence_respects_max_iterations_budget`
8. `test_convergence_respects_cost_budget`
9. `test_loop_resumable_from_saved_state`
10. `test_human_directive_incorporated_in_next_formulate` — `@curio focus on X` works
11. `test_loop_produces_langfuse_trace_per_iteration`

**Done:** `axi research start "test topic"` on a small corpus converges within 4 iterations and produces a synthesis file. `axi research resume <id>` continues from saved state.

### Slice 5: Hello Bronze (~1d)

**Behavior:** SeaweedFS deployed on Rascal, chat traces written to bronze as raw JSONL partitioned by date. Queryable via DuckDB.

**Scope:**
- Data platform scaffolding: `axiom/src/axiom/infra/data_platform/`
- SeaweedFS client wrapper (S3 API)
- Bronze ingestion worker (listens to trace events, writes JSONL to SeaweedFS)
- DuckDB query helper for bronze parquet/JSONL files
- SeaweedFS deployment added to Rascal Docker Compose

**Tests:**
1. `test_seaweedfs_client_put_and_get_object`
2. `test_bronze_ingest_from_trace_event` — trace event → JSONL line in SeaweedFS
3. `test_bronze_partitioning_by_date` — today's events in today's file
4. `test_duckdb_query_bronze_returns_rows` — DuckDB over SeaweedFS works
5. **Integration:** run `neut chat` on Rascal, query DuckDB for today's traces, find them

**Done:** Chat on Rascal → trace emitted → visible in bronze via DuckDB within 10 seconds.

### Slice 6: Hello Medallion (~2-3d)

**Behavior:** Silver cleans and structures bronze. Gold aggregates. Finding promotion from gold triggers RAG update. Dagster orchestrates.

**Scope:**
- Silver transform job (bronze raw → structured traces with attribution)
- Gold aggregate job (traces → research findings candidates)
- Iceberg table format for silver and gold
- Dagster DAG: bronze → silver → gold → RAG
- RAG promotion writer (gold findings → RAG corpus with attribution chain)

**Tests:**
1. `test_silver_transforms_raw_to_structured`
2. `test_silver_deduplicates_by_trace_id`
3. `test_gold_aggregates_per_research_loop`
4. `test_gold_identifies_promotion_candidates` — high-confidence findings flagged
5. `test_rag_promotion_includes_attribution_chain` — promoted finding has contributor + verifier chain
6. `test_dagster_dag_runs_end_to_end` — full bronze→silver→gold→RAG in one orchestrated run
7. `test_rag_search_returns_newly_promoted_finding` — promoted content is searchable immediately

**Done:** A completed research loop produces findings in gold; a manual approval (or auto-approval for high-confidence) promotes into the course RAG; next chat query retrieves the newly promoted content.

### Slice 7: Hello Federation (~2-3d)

**Behavior:** Two real nodes exchange research digests over a real network. Cross-pollination visible. Signed messages, verified, rate-limited, quarantine enforced.

**Scope:**
- Ed25519 keypair generation per platform identity, per node, per context (ADR-020)
- Signed federation message envelope (3-layer verification chain)
- Research digest publisher (per curio-research-loop.md §8)
- Digest receiver + validator
- Quarantine state machine
- Per-peer rate limiter
- Local reputation primitives (counter: evals passed, contradictions, complaints)

**Tests:**
1. `test_keypair_generation_and_signature_verification`
2. `test_three_layer_signed_message_verifies`
3. `test_signed_message_rejected_when_any_layer_tampered`
4. `test_digest_publishes_on_heartbeat`
5. `test_digest_received_and_stored_per_peer`
6. `test_peer_in_quarantine_findings_not_auto_promoted`
7. `test_rate_limit_circuit_breaker_after_violations`
8. `test_access_tier_filter_strips_restricted_content_from_federated_digest`

**Done:** Two nodes (Rascal + laptop, or two Rascal-class hosts) exchange digests every 5 minutes. Node A sees Node B's public findings. Quarantine prevents immediate trust.

### Slice 8: Hello Eval Uplift (~1-2d)

**Behavior:** Demonstrate that federation + RAG promotion measurably improves local eval scores. The proof-of-compound-knowledge.

**Scope:**
- Eval suite for a specific domain (~20 questions)
- Baseline run against Node A alone (before federation)
- Run Node B's research loops on complementary topics, promote findings, replicate to Node A
- Re-run eval suite against Node A after federation
- Compare scores

**Tests:**
1. `test_baseline_eval_run_records_scores` — N/M pass rate captured
2. `test_post_federation_eval_shows_improvement` — after federating in Node B's findings, Node A's pass rate on affected questions improves
3. `test_eval_compare_renders_comparison_table` — `axi eval compare` shows before/after delta per case
4. **Integration proof:** baseline ≤ 14/20, post-federation ≥ 17/20 on a targeted suite

**Done:** A single command demonstrates federation improves local answer quality on domain questions. This is the Figure 1 of any research paper we write.

---

## Real-Node Federation Test Protocol

For Slice 7 and 8, "real nodes" means actual separate processes with real network, real keys, real corpora. No mocks for the federation layer.

**Candidate topologies:**

**Option A (simplest, recommended):** Two Rascal-class hosts, one being Rascal itself, one being a second machine on the UT network or a cloud VM. Federation occurs over TLS. Both run Server profile.

**Option B:** Rascal + Ben's laptop (Edge profile) over the internet. Tests profile heterogeneity but adds laptop availability as a risk factor.

**Option C:** Rascal + Rascal-inside-a-different-K3D-namespace. Isolates federation at the process level without a second physical machine. Simpler to set up but doesn't prove wire-level federation.

**Recommended:** start with Option C for development speed, graduate to Option A before calling Phase 0 done.

**Federation test matrix:**

| Scenario | What's tested | Success metric |
|----------|---------------|----------------|
| Clean startup, both nodes empty | Basic handshake, keypair exchange, mutual trust establishment | Both nodes report peer as "quarantined" with correct identity |
| Node A runs a research loop, converges, promotes finding | Local promotion works standalone | Finding appears in Node A's gold, then RAG |
| Digest exchange after Node A promotes | Federation digest includes Node A's finding | Node B receives digest with signed finding reference |
| Node B requests the finding, validates attribution chain | Content retrieval respects access tier, signatures verify | Finding arrives at Node B with intact attribution |
| Node A runs a baseline eval | Baseline score captured before federation | 14/20 (example number) |
| Node B promotes complementary findings, federates to A | Cross-pollination | Node A's RAG includes B's findings, post-quarantine |
| Node A re-runs the eval | Score improves | 17/20+ |
| Network partition during digest exchange | Graceful degradation | Retry on reconnect, no corruption |
| Tampered message injected | Signature rejection | Message rejected, peer penalty recorded |

---

## First Failing Test

The very first thing to write — red test that drives the first commit on this branch:

```python
# axiom/src/axiom/infra/tracing/tests/test_provider_protocol.py

def test_trace_provider_protocol_is_importable():
    """Trace provider protocol exists as a typing.Protocol."""
    from axiom.infra.tracing import TraceProvider
    assert TraceProvider is not None


def test_trace_provider_has_required_methods():
    """Protocol defines the minimum method set we need for classroom and evals."""
    from axiom.infra.tracing import TraceProvider
    required = {"start_trace", "log_generation", "log_retrieval", "score", "flush"}
    members = {name for name in dir(TraceProvider) if not name.startswith("_")}
    assert required.issubset(members), f"missing: {required - members}"
```

These tests fail because `axiom.infra.tracing` does not yet exist. The first implementation commit creates `axiom/src/axiom/infra/tracing/__init__.py` with the `TraceProvider` Protocol and makes the tests green.

---

## Estimated Schedule

| Slice | Days (focused) | Cumulative |
|-------|:-------------:|:----------:|
| 1. Hello Trace | 1 | 1 |
| 2. Hello Evals | 1-2 | 2-3 |
| 3. Hello WebUI | 1-2 | 3-5 |
| 4. Hello Research Loop | 2-3 | 5-8 |
| 5. Hello Bronze | 1 | 6-9 |
| 6. Hello Medallion | 2-3 | 8-12 |
| 7. Hello Federation | 2-3 | 10-15 |
| 8. Hello Eval Uplift | 1-2 | 11-17 |

**Total: 11-17 focused-work days.** Calendar time depends on parallelism — slices 3 and 5 can run in parallel with others if deployment work is independently threaded. Slices 7 and 8 require all prior slices complete.

Each slice ends with a push-able commit + green tests. No branch should ever be left in a half-working state.

---

## Deferred Decisions (Non-Blocking)

These come up during Phase 0 but don't block it. Flagged here; revisited when they actually block progress or Phase 0 completes.

| Decision | When to revisit |
|----------|-----------------|
| ADR-022: Federation identity root authority (self-sovereign vs. InCommon vs. domain-based) | When we federate with an external institution (not UT ↔ UT) |
| Full reputation algorithm (past Phase 0 primitives) | After observing real federation behavior in Slice 7-8 |
| Alumni harvest bundle format specifics | v3 scope, post-Phase 0 |
| LTI 1.3 implementation | Prague-specific, after Slice 6 if Canvas/Moodle required before class |
| Multi-context workspace UI in Open WebUI | When users ask for it — initially CLI-only context switching |
| Key rotation protocol | When first key rotation is needed operationally |
| Dagster vs. lighter orchestrator (Prefect? Temporal?) | If Dagster is too heavy for Edge/Workstation profiles |

---

## Exit Criteria for Phase 0

Phase 0 is complete when ALL of:

1. All 8 slices end green on the branch
2. Rascal is running as a Server-profile deployment with full stack
3. Open WebUI is functional for a test cohort of 2-3 people (Ben, Soha at minimum)
4. A two-node federation test demonstrates measurable eval uplift (Slice 8 integration proof)
5. The combined test suite runs in CI on every PR, including real-infrastructure integration tests (skippable locally with `--no-integration`)
6. The branch is merged to main

At that point, we have proven:
- Classroom infrastructure works
- Data platform flows end-to-end from chat to RAG
- Federation moves findings between nodes
- Evals improve as knowledge accumulates

And we have the foundation for Prague MVP (v1) delivery.

---

*See also: ADR-019 Node Profiles, ADR-020 Federation Identity, ADR-021 Threat Model, prd-classroom.md, prd-evals.md, architecture/curio-research-loop.md.*
