Metadata-Version: 2.4
Name: agent-wiki
Version: 0.1.0
Summary: Toolkit for building agent-maintained Obsidian-style wikis — link management, linting, document conversion, and agent coordination
License-Expression: MIT
Keywords: agent,wiki,obsidian,markdown,knowledge-base,ai-agents
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Text Processing :: Markup :: Markdown
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymupdf>=1.24.0
Requires-Dist: pymupdf4llm>=0.0.17
Requires-Dist: Pillow>=10.0.0
Provides-Extra: future
Requires-Dist: python-docx>=1.0.0; extra == "future"
Requires-Dist: python-pptx>=0.6.23; extra == "future"
Requires-Dist: openpyxl>=3.1.0; extra == "future"
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Dynamic: license-file

# agent-wiki

**Toolkit for building LLM-maintained wikis.** Handles the plumbing — link management, linting, file operations, document conversion, and agent coordination — so LLMs focus on content.

Based on the [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern by Andrej Karpathy: instead of RAG (re-deriving knowledge on every query), the LLM **incrementally builds and maintains a persistent wiki** — a structured, interlinked collection of markdown files. The wiki is a compounding artifact: cross-references are already there, contradictions already flagged, synthesis already current. You curate sources and ask questions; the LLM does the bookkeeping.

## Install

```bash
pip install agent-wiki
```

## Quick Start

```python
from agent_wiki import WikiRoot

wiki = WikiRoot("/path/to/wiki")

# Health check — find broken links, missing backlinks, orphan pages
issues = wiki.lint()

# Move a page and auto-update every [[link]] across the wiki
wiki.move("topics/Old Name.md", "topics/New Name.md")

# Convert a PDF to structured markdown with images
wiki.convert_pdf("paper.pdf", "processed/paper.md", max_dpi=150)

# Find all pages linking to a topic
wiki.backlinks("Sand Injectites")

# Search for a concept across all pages
wiki.find_references("polygonal fault")

# Wiki statistics
wiki.stats()
```

## CLI

Every operation is also available from the command line — designed for AI agents calling via Bash.

```bash
# Lint
agent-wiki --root wiki/ lint
agent-wiki --root wiki/ lint --json          # structured output for agents

# File operations (auto-updates all links)
agent-wiki --root wiki/ move old.md new.md
agent-wiki --root wiki/ merge source.md target.md
agent-wiki --root wiki/ rename "Old Name" "New Name"

# Document conversion
agent-wiki --root wiki/ convert pdf paper.pdf processed/paper.md --max-dpi 150

# Search
agent-wiki --root wiki/ backlinks "Page Name"
agent-wiki --root wiki/ find-references "some concept"
agent-wiki --root wiki/ stats
```

Use `--json` on any command for machine-readable output.

## Features

### Link Management

Obsidian-style `[[wiki-links]]` with full support for `[[target|display text]]`. The library builds a link graph, resolves links by filename stem (Obsidian's shortest-unique-path matching), and rewrites links automatically when files move.

```python
from agent_wiki.links import parse_links, find_backlinks, rewrite_links

links = parse_links("See [[Topic A]] and [[Topic B|related topic]]")
# → [WikiLink(target="Topic A", ...), WikiLink(target="Topic B", display="related topic", ...)]

rewrite_links(text, "Old Name", "New Name")
# → updates [[Old Name]] → [[New Name]], preserves [[Old Name|display]] → [[New Name|display]]
```

### Linting

Automated wiki health checks that catch real problems:

| Check | Severity | What it detects |
|-------|----------|-----------------|
| Broken links | error | `[[Target]]` where no file matches |
| Missing backlinks | warning | Topic lists source but source doesn't link back |
| Orphan pages | warning | Pages with zero inbound links |
| Broken breadcrumbs | error | Navigation chain has dead links |
| Missing frontmatter | error | Required YAML fields missing per page type |
| Missing sections | warning | Topic without `## Sources`, source without `## Topics` |
| Dispute chronology | warning | Disputed claims with dates out of order |
| Split candidates | info | Pages exceeding 500 lines |

```python
issues = wiki.lint()
for issue in issues:
    print(f"[{issue.severity.value}] {issue.file}: {issue.message}")
```

### File Operations

Move, rename, or merge pages — all `[[wiki-links]]` across the entire wiki are updated automatically.

```python
# Rename a page — finds it by name, renames file, updates all references
wiki.rename("Old Topic Name", "New Topic Name")

# Merge two pages — appends content, redirects all links, deletes source
wiki.merge("sources/duplicate.md", "sources/canonical.md")
```

### Document Conversion

PDF to structured markdown using [pymupdf4llm](https://github.com/pymupdf/RAG) — proper headings, paragraphs, tables, and extracted images. Not flat text.

```python
wiki.convert_pdf(
    "paper.pdf",
    "processed/paper.md",
    max_dpi=150,           # image resolution cap
    extract_images=True,   # images saved to img/ subfolder
)
```

Stubs for `.docx`, `.pptx`, `.xlsx` conversion are included for future implementation.

### Kanban Agent Pipeline

A filesystem-based kanban system for coordinating multiple AI agents. Agents communicate through task cards — lightweight markdown files that move between columns.

```python
from agent_wiki.kanban import create_card, claim, complete, list_cards

# Create a task card (auto-generated by kanban_process)
create_card(
    source_file="raw/paper.pdf",
    processed_file="processed/paper.md",
    kanban_dir="kanban/backlog/",
    agent="reader",
)

# Agent claims work (atomic move — prevents race conditions)
card = claim("kanban/backlog/paper.md", "kanban/processing/")

# Agent finishes — move to next stage
complete("kanban/processing/paper.md", "kanban/review/", agent="writer")
```

**Batch processing with `kanban_process`:**

```python
# Scan for new files, convert, create task cards — one call
cards = wiki.kanban_process(
    input_dir="raw/",
    output_dir="processed/",
    completed_dir="./completed",      # relative to input_dir
    kanban_dir="kanban/backlog/",
)
# → converts new PDFs, moves originals to raw/completed/, creates task cards
```

**Pipeline pattern:**

```
raw/paper.pdf
  → kanban_process() converts + creates card
    → reader agent claims → writes source page → passes to writer
      → writer agent claims → writes topic pages → passes to orchestrator
        → orchestrator reviews → approves to wiki or sends back with actions
```

No database. The filesystem is the state. Agents coordinate by moving files.

## Project Layout

agent-wiki works with any Obsidian-compatible wiki. A typical project:

```
my-wiki-project/
├── raw/                    # source documents (immutable)
│   └── completed/          # originals moved here after processing
├── wiki/                   # ← this is your WikiRoot
│   ├── processed/          # converted markdown (auto-generated)
│   ├── sources/            # source pages (one per paper)
│   ├── topics/             # topic pages (synthesized knowledge)
│   ├── kanban/             # task cards for agent coordination
│   │   ├── backlog/
│   │   ├── processing/
│   │   ├── review/
│   │   └── done/
│   ├── index.md
│   └── log.md
├── instructions/           # agent prompts and workflow docs
└── agent-wiki.yaml           # optional config
```

## Configuration

Optional `agent-wiki.yaml` at the project root:

```yaml
root: wiki/
kanban: wiki/kanban/
processed: wiki/processed/
raw: raw/
completed: raw/completed/
```

Or configure programmatically:

```python
wiki = WikiRoot("wiki/", kanban_dir="wiki/kanban/", processed_dir="wiki/processed/")
```

## The Pattern

The LLM Wiki pattern has three layers:

1. **Raw sources** — your curated documents. Immutable. The LLM reads but never modifies.
2. **The wiki** — LLM-generated markdown. Summaries, topic pages, cross-references. The LLM owns this entirely.
3. **The schema** — instructions that tell the LLM how the wiki is structured and what workflows to follow.

Three operations:

- **Ingest** — process a new source into the wiki. Creates a source page, updates topic pages, maintains cross-references.
- **Query** — answer questions from the wiki. Good answers get filed back as new pages.
- **Lint** — health-check the wiki. Find broken links, orphan pages, contradictions, missing cross-references.

The human curates sources and asks questions. The LLM does the bookkeeping.

For the full pattern description, see [Andrej Karpathy's LLM Wiki gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).

## Requirements

- Python 3.12+
- Dependencies: `pymupdf`, `pymupdf4llm`, `Pillow`

## License

MIT
