---
name: kb-ingest
title: "KB Ingest"
description: "Classify incoming files, extract content, create source summaries for the knowledge base"
category: core
status: active
model_tier: capable
triggers:
  - "ingest"
  - "new file in inbox"
  - "process sources"
# Tools granted to any subagent that runs this skill. ``run_wiki_job``
# is the canonical entry point — it runs snapshot → ingest → compile →
# reflect → lint for a given KB (or all KBs when no slug is passed).
# ``query_knowledge`` lets the librarian check for existing entries
# before making create-vs-update decisions during classification.
# ``list_subagents`` is here so the librarian can delegate crosslinking
# or compilation to the editor when the workload warrants it.
tools_granted:
  - run_wiki_job
  - query_knowledge
  - list_subagents
# Few-shot examples so small local LLMs know exactly which tool to call.
# The agent dispatched with this skill sees ONLY the tools_granted set
# (not the full access tier), so the examples must reference real tool
# names from that set. run_wiki_job is the primary entry point.
examples:
  - input: |
      Run skill `kb-ingest` with the following arguments:
      {"slug": "general"}
    output: |
      I'll process the inbox for the "general" knowledge base by running the full wiki pipeline.
      [Calls tool: run_wiki_job with {"kb": "general"}]
      The wiki job completed: ingested 3 new sources, compiled 2 wiki articles, no lint issues found.
  - input: |
      Run skill `kb-ingest` with the following arguments:
      {"slug": "research"}
    output: |
      Processing the "research" KB inbox now.
      [Calls tool: run_wiki_job with {"kb": "research"}]
      Done: 1 new paper ingested, 1 article compiled. The source was classified as a research paper with 4 key entities extracted.
language:
  # Output language policy for summaries, concepts, and extracted prose.
  #   "source"   — match the detected source language (default)
  #   "user"     — match the user's UI locale
  #   "english"  — always English
  #   "<code>"   — specific ISO 639-1 code (e.g., "sv", "de", "fr")
  #
  # This can be overridden per KB via SCHEMA.md `language_policy` or
  # per file via frontmatter `language_policy` on the source file itself.
  output_policy: source
  # Fallback when detection fails, is too uncertain, or returns an
  # unsupported language.
  fallback_language: en
  # Minimum text length (chars) before attempting detection.
  min_detection_chars: 50
  # Minimum langdetect confidence (0.0-1.0) to accept a detection result.
  min_confidence: 0.6
---

## Instructions

Process new files dropped into a KB inbox. For each file:

1. **Classify** — determine source type: article, paper, note, image, data, code, meeting-notes
2. **Extract** — pull out key information:
   - Summary (2-3 sentences)
   - Key concepts (comma-separated)
   - Named entities (people, organizations, places)
   - Contradictions with existing wiki content (if any)
   - Suggested wiki updates (CREATE entity/concept, UPDATE existing)
3. **Create source summary** — wiki page in `sources/` with:
   - Full frontmatter (title, entity_type, category, taxonomy_path, source_file, vault_source)
   - Summary, key takeaways, contradictions section
   - Link back to original file
4. **Move to library** — organized by type and date (`library/{type}/{YYYY-MM}/`)

## Quality Rules

### What to Skip (file-level)
Skip files that don't contain useful knowledge:
- Time reports, timesheets, time tracking
- Templates (day template, meeting template)
- Untitled / empty files
- Files under 50 characters of real content

### What Makes an Entity Worth a Page
Only CREATE wiki pages for entities that are SIGNIFICANT to the source:

**Significance test:** would someone search for this to understand the domain?
- YES: "Bowtie Analysis", "FMEA", "Sarah Chen" — core to the domain
- NO: "hexadecimal notation", "bugs", "court cases" — generic, passing mentions

**Rules:**
- Maximum 5 CREATE lines per source file
- Prefer UPDATE to existing pages over creating new ones
- Every CREATE must include a REASON explaining why it deserves a page
- People: only create if they have a described role or contribution (not just a name mention)
- Organizations: only create if their function is described
- Concepts: only create if the source explains what it is (not just mentions it)

### Entity Type Guidelines
| Type | Create when... | Skip when... |
|------|---------------|--------------|
| person | Named with role, affiliation, or contribution | Just a name in passing |
| organization | Described with function or relationship | Just a name in passing |
| concept | Explained with definition or usage | Generic term everyone knows |
| project | Has goals, timeline, or deliverables | Just a project name mention |
| event | Has date, participants, outcomes | "We had a meeting" with no details |

## Rules

- Every claim must trace to its source — no invented content
- NEVER FABRICATE URLS, LINKS, OR METADATA. Do NOT invent source_url, publisher, author, or DOI values. If the file has a real URL in frontmatter (source: or source_url:), use it. If not, the source is a local file — leave the URL empty.
- Only assert relationships STATED in the source. If a person is mentioned without a role, extract the person but NOT a fabricated role.
- Tag with the best EXISTING taxonomy path — never create new categories
- Track vault_source for files originating from the user's vault
- Images go through vision LLM for description

## Dedup Guards (system-enforced)

- **Content hash**: skip files whose hash matches an already-processed file
- **Cross-KB**: skip files whose hash exists in any other KB's processed state
- **Filename dedup**: same content under different filenames is detected and skipped
- **Updated files**: when same stem appears with different hash, old source summary + produced pages are cleaned up
- **Index dedup**: `_index.md` won't double-list the same source
- **Max retry**: files that fail LLM extraction 5 times are permanently skipped (per session)

## System Prompt

Classify this source document and extract information.

Source file: {filename}
Source type: {source_type}
Type-specific focus: {type_focus}

IMPORTANT: Web-clipped articles often start with ads, navigation bars, cookie notices,
or sidebar content that was captured along with the main article. Always use the SOURCE
METADATA (title, description, URL) provided in the system prompt to determine the article's
TRUE topic. Do NOT use the first line of body text as the topic if it contradicts the metadata.

KB Schema:
{schema}

Existing wiki index (what's already in the KB):
{index}

RULES:
- Every fact must trace to the source text. Do NOT invent information.
- NEVER fabricate URLs, publisher names, author names, or DOIs. If no URL is in the source metadata, leave it empty.
- Only extract relationships (roles, affiliations) that are EXPLICITLY STATED in the source text. Do NOT infer or guess.
- If something is unclear, write [TODO: clarify from source].
- Note any contradictions with the existing wiki index.
- The frontmatter title and description are AUTHORITATIVE for identifying the article topic.
- Ignore ad banners, navigation text, and sidebar content that may appear at the start of the body.

Produce:

```summary
(3-5 sentence summary of the source, citing specific details)
```

```concepts
- concept1
- concept2
```

```entities
- person: Name — role/description
- organization: Org Name — what they do
```

```contradictions
(any facts that contradict existing wiki content, or "none found")
```

```wiki_updates
List ONLY entities and concepts that are SIGNIFICANT to this source — not every noun mentioned.

Significance test: would someone search for this concept to understand the source's domain?
- YES: "Bowtie Analysis", "FMEA", "Sarah Chen" — core to the domain, worth a page
- NO: "hexadecimal notation", "bugs", "court cases", "time management" — generic, passing mentions

CREATE sparingly. Prefer UPDATE to existing pages. Maximum 5 CREATE lines per source.

Format:
- CREATE entities/person-slug: Full name and role | REASON: why this person matters
- CREATE entities/org-slug: Organization name | REASON: significance to the domain
- CREATE concepts/topic-slug: What this concept is about | REASON: why it deserves its own page
- UPDATE concepts/existing-slug: What new info to add from this source
```
