Knowledge Compiler#

Agents do research, pull data, write reports—the wiki connects these outputs. The more you use the agent, the more it understands your domain.

Runtime Dependencies & Permissions#

DependencyRequired?DescriptionPython 3.8+✅ Requiredschema.py (frontmatter validation), store.py (SQLite data management), build_index.py (FTS5 indexing) are Python scripts. Uses only stdlib ( sqlite3, json, pathlib) + pydanticpydantic✅ RequiredFrontmatter validation in schema.py. pip install pydanticFilesystem write✅ RequiredCreates and edits Markdown, SQLite, .obsidian/ config under .wiki/{topic}/. Will confirm location with user before first .wiki/ creation****WebSearch / WebFetch❌ OptionalNeeded for active mode (agent searches for materials autonomously). Not needed for passive mode (user provides files)External validator (MCP)❌ OptionalOnly called during lint when wiki declares a validator. Silently skipped when unreachable, zero impact. No user credentials needed — Mcp-Session-Id is standard MCP protocol session handshake, handled automatically by the agentDomain data MCP❌ Optionaldeep-dive and active ingest can use domain data MCP for enhanced search quality. Falls back to WebSearch when absent

Core promise: Passive mode (user provides files → agent compiles) only requires Python 3 + filesystem access, zero network dependencies. All network calls are optional enhancements, and the environment check on first use will inform the user of available capabilities.

Quick Start#

text

User: /auto-wiki recall personal-pension
Agent: [Scan .wiki/personal-pension/ → Read index.md → Load data.db summary]
Agent: Entered recall mode. Current wiki: 22 pages / 8 data points / 2 contested.
       Subsequent questions will be answered by checking wiki first.

User: What are the reasons for low participation rates?
Agent: [Read wiki pages enrollment-friction, tax-incentive-effect, etc.]
Agent: According to 6 sources accumulated in wiki... (cite specific pages and data)
       ⚠️ Note: Tax incentive effects are contested (77.8% vs 25%), see [[participation-willingness]]

text

User: ingest this industry report into XX wiki
Agent: [Read report → Search existing wiki → Compare old vs new → Update 3 pages, create 1 page]
Agent: Ingested into XX wiki: updated 3 pages (list), created 1 page (list)

Core Philosophy#

Agents help you research, write reports, and pull data every day—but then forget everything. The next time you ask about the same domain, it starts from scratch.

This skill solves one thing: giving agents a knowledge base that accumulates over time.

It's not RAG (retrieving from documents on each query), but compilation—after reading source files, the agent writes key information into existing wiki pages, comparing with old knowledge, merging, and flagging conflicts. Before executing any task next time, it reads the wiki and works from the accumulated foundation.

Four Modes#

ModeTriggerWhat Agent Doesrecallrecall / recall {topic}Load wiki context, answer all subsequent questions by checking wiki firstingestUser provides source file or textRead source → Search wiki → Compare old vs new → Update/create pages → Update indexqueryUser asks a question (one-time)Read index → Find relevant pages → Synthesize answer → Archive valuable analysislintUser says "check wiki"Scan all pages → Merge duplicates → Archive outdated → Report conflicts and healthdeep-divedeep-dive / "level up"Run Coverage lint → Present gap report → User confirms → Search + ingest to fill gaps

deep-dive is not a 5th independent mode—it's a combined pipeline of lint (Coverage) and ingest (with search tools). Requires search tools (active mode).

Recall mode vs query difference: query is one-time operation (ask one question, check wiki once). Recall mode is continuous state—after entering, every question in this conversation goes through wiki first.

---

Recall Mode#

Entering#

Triggered when user says /auto-wiki recall or /auto-wiki recall {topic}.

Agent executes:

1. Scan .wiki/ directory, list available wiki topics

2. If user specified topic → Load that wiki; If not specified → List optional topics for user to choose

3. Read index.md → Get all page list and structure

4. Read data.db summary → python references/store.py dump .wiki/{topic}/, get data point count, relation count, contested count

5. Report to user: text

   Entered recall mode: {topic}
   

• Pages: {N} (sources: X, entities: Y, concepts: Z)
• Data points: {N} | Relations: {N} | Contested: {N} Subsequent questions will be answered by checking wiki first. Say "exit recall mode" to return to normal.

### Response Flow

After entering recall mode, upon receiving each user question:

1. **Extract keywords from question** (entity names, concept names, metric names)
2. **Match in index.md** for relevant pages (title + description)
3. **Query data.db for relevant data points**:
   sql

   ```sql
   SELECT * FROM data_points WHERE field LIKE '%keyword%' OR page_slug LIKE '%keyword%'

1. Read matched wiki pages (usually 2-5), expand one level along wikilinks

2. Synthesize answer, must:

   • Cite specific pages: [[slug]]
   • Cite specific data: value + unit + period + source
   • If involving contested info, proactively label
   • If wiki info insufficient, clearly state "no accumulation in this aspect in wiki, suggest ingesting XX"

3. Do not fabricate information not in wiki. Better to say "don't know" than pretend wiki has it

Exit#

Exit when user says exit recall, switches to other operations (ingest/lint), or starts a new topic.

---

Execution Flow#

Phase 0: Identify Research Topic and Ontology Type#

After receiving user input, determine three things: operation type, target wiki, ontology type.

User InputOperationTarget WikiOntology Type"Help me organize this report" + fileingestInferred from content, or ask userdomain"ingest into personal pension" + fileingestpersonal-pensiondomain"Research Charlie Munger" + materialsingestcharlie-mungercognitive"What is personal pension participation rate?"queryInferred from question, or ask user—"Check the pension wiki"lintpersonal-pension—

Ontology type determines wiki page structure and collection strategy:

Ontology TypeResearch TargetPage FocusReferencecognitivePerson (mental models, decision patterns)MentalModel, Heuristic, Value, StylePatternreferences/ontology-types/cognitive.mddomainDomain (institutions, systems, metrics)Entity, Concept, Metricreferences/ontology-types/domain.mdgeneralNeither of the aboveDefault entity/concept structure—

A wiki has only one type. If research spans both person and domain (e.g., "Munger's investment framework applied to enterprise annuity"), it belongs to two wikis, with cross-wiki query for synthesis. Don't mix cognitive and domain page structures in one wiki.

If wiki directory doesn't exist, first confirm creation location with user (default .wiki/{topic}/ under current repository root), then create initial structure per references/storage-spec.md (including meta.yaml, index.md template, log.md template). Suggest user add .wiki/ to .gitignore if not already present.

Domain seed: If the target domain has a corresponding seed file ( seeds/{name}.md), declare seed: {name} in meta.yaml. Seeds provide standard terminology, relationship templates, and anti-confusion rules, letting wikis start from a normalized foundation. Domains without seeds grow freely — both paths work. Seeds are community-contributable plugins — anyone can write a markdown file for their vertical domain. See references/seed-ontologies.md.

On first use, perform environment check (see references/source-validation.md), informing user of available capabilities (passive mode vs active mode).

Reference Loading Strategy#

Don't read all references at once. Load on-demand by operation type:

OperationRequiredRead on First UseRead When Tools Availableingestingest-protocol.md, wiki-format.md, schema.py``storage-spec.md (if wiki doesn't exist), seed-ontologies.md + seeds/{name}.md (if seed declared in meta.yaml)fact-check.md, source-validation.mdqueryquery-protocol.md——lintlint-protocol.md, schema.py—validators/{name}.md (if seed declared validator)deep-divelint-protocol.md, ingest-protocol.md, source-validation.md, wiki-format.md, schema.py``storage-spec.md (if wiki doesn't exist)fact-check.md

Not needed: scaling.md only relevant when page count > 500; ontology-types/ only when creating new wiki and need to determine type.

Phase 1: Ingest (Knowledge Compilation)#

This is the core operation. Detailed protocol in references/ingest-protocol.md.

Brief flow:

1. Read source file, extract key information

2. Validate key data (if tools available)—see references/fact-check.md

3. Write source summary page ( sources/{date}-{slug}.md)

4. Search wiki for existing related pages (read index.md, grep key entity names)

5. Compare old vs new page by page:

   • New info supports existing conclusion → add citation, boost confidence
   • New info overturns existing conclusion → write value to data.db (old value auto-enters history table), rewrite body analysis
   • New info conflicts and can't determine → present both views, confidence → contested

6. Create new pages (only for entities/concepts not yet in wiki)

7. Update index.md + append log.md

8. Schema validation—run python references/schema.py {page.md} on all created/modified pages to ensure frontmatter compliance. Fix immediately if failed

After ingest, report to user:

text

Ingested into {topic} wiki:
- Created: {N} pages (list)
- Updated: {N} pages (list + brief reason)
- Conflicts: {N} (list conflict points)
- Validation: {N} pages all passed / {M} pages have issues (list)

Phase 2: Query (Knowledge Retrieval)#

Detailed protocol in references/query-protocol.md.

1. Read index.md, identify pages relevant to question

2. Read matched pages + follow wikilinks to related pages (one level)

3. Synthesize answer based on page content, cite source pages: text

   Based on {N} source files accumulated in wiki:
   ... analysis content ...
   Sources: [[alpha-corp]], [[2026-policy-doc]]
   
   

4. If involving contested info, clearly label the conflict

5. If answer contains valuable new analysis, prompt user to archive

If wiki info insufficient, clearly state the gap:

text

Wiki information on XX is insufficient. Currently only 2 relevant source files.
Suggest ingesting more materials about XX.

Phase 3: Lint (Knowledge Governance)#

Detailed protocol in references/lint-protocol.md (7 checks + health report format).

Lint has two levels:

LevelTriggerCheck ItemsCostStructural (default)lint / check wikiValidation, Orphan, Broken Link, StalenessFull scan, deterministicSemantic (on-demand)deep lint / check conflictsContradiction, Duplication, CoverageAgent semantic understanding, scope-controlled

1. Structural: Auto-scan all pages, fix format, broken links, orphans, staleness
2. Semantic (user triggered): Detect conflicts, duplicates, coverage gaps. Wiki < 50 pages = full scan, 50-200 pages = only last 30 days' ingest-touched pages, > 200 pages = user must specify scope
3. Report health:

text

Wiki Health Report: {topic}
- Total pages: 42 (sources: 12, entities: 15, concepts: 10, analyses: 5)
- Health: Good
- Structural fixes: Fixed 1 broken link, archived 1 stale page
- [Semantic] Pending human review: 2 conflicts (list)
- Suggestion: Few source files in XX domain (only 1), recommend supplement

Phase 4: Deep-Dive (Knowledge Completion Pipeline)#

deep-dive = lint(Coverage) + ingest(search-fill). Not an independent mode, but a combined pipeline.

Prerequisite: Requires search tools (active mode). Without search tools, only outputs gap report without auto-filling. Prompts user to manually ingest.

Flow:

text

1. Run lint Coverage check (5-category gap detection)
   → Output structured Gap Report (see lint-protocol.md)

2. Present Gap Report, request user confirmation
   → User can: accept all / select subset / limit scope / cancel
   → This step cannot be skipped — prevents unsupervised batch writes

3. For each confirmed gap, execute from-lint ingest flow
   → Search → User confirms source → Standard ingest
   → See From-Lint section in ingest-protocol.md

4. Output completion report: filled / unable to fill / recommendations

Trigger words: deep-dive, deep dive, fill gaps, research gaps, level up

Example:

text

User: deep-dive treasury-futures
Agent: [Running Coverage lint...]
Agent: Found 6 knowledge gaps:
       1. [high] page_missing: stock-bond-correlation (referenced by 4 pages)
       2. [high] concept_missing: basis (mentioned in 5 entity pages)
       3. [medium] single_source: treasury-futures-basics (only 1 source)
       ...
       Which to fill? (all / select numbers / cancel)

User: 1, 2

Agent: [Searching "stock bond correlation treasury futures"...]
Agent: Found 2 candidate sources:
       - [authoritative-secondary] CICC Fixed Income "Stock-Bond Correlation Analysis" ← Recommended
       - [secondary] Blog post ← Skip (blacklisted channel)
       Confirm using CICC report?

User: confirm

Agent: [Execute standard ingest → Create concepts/stock-bond-correlation.md]
Agent: Completion done. Created 2 pages, updated 0 pages, 1 gap unable to fill (suggest providing materials manually).

---

Wiki Page Format#

See references/wiki-format.md for details. Brief summary:

• Each page is markdown with frontmatter (title, type, created, updated, sources, confidence)
• 5 page types: source / entity / concept / analysis / mental-model
• Use [[slug]] for inter-page links
• index.md is the directory, log.md is the operation log

Ontology Type Reference#

When research target is a person, see references/ontology-types/cognitive.md for collection strategy—page types focus on mental models, heuristics, value systems, expression styles.

When research target is a domain, see references/ontology-types/domain.md for collection strategy—page types focus on institutional entities, system concepts, quantitative metrics.

Both share the same wiki infrastructure (ingest/query/lint), differing only in page classification and collection emphasis.

Vertical Domain Adaptation#

The skill core is a domain-agnostic compilation engine. Vertical domain expertise is injected through two plugin layers:

LayerCarrierPurposeRequired?Seedseeds/{name}.mdCold-start vocabulary: standard terms, relationship templates, anti-confusion rulesOptionalValidatorvalidators/{name}.mdRuntime logical validation: relationship legality, required relation completenessOptional

Without plugins, wikis grow freely — suitable for exploratory research. With plugins, wikis start from industry standards with normalized naming, clear relationship structures, and detectable logic gaps.

Community-contributable: Write a seed file (markdown) for your vertical domain — declare 20-50 core terms and anti-confusion rules, and wikis in that domain start from a normalized foundation.

Currently available:

• seeds/fibo-pensions.md — Enterprise annuity / pensions (based on FIBO standard)
• validators/fibo-mcp.md — FIBO SPARQL logical validation (627K inferred triples)

What This Skill Doesn't Do#

• No vector retrieval. Small scale uses index + grep, large scale uses SQLite FTS5 + BM25 (see references/scaling.md). Vector retrieval is left to platform-level tools.
• No multi-user collaboration. Wiki directory is local files, one wiki per user.
• No replacement for professional data tools. Domain data acquisition uses corresponding MCP/tools; this skill only receives their outputs and compiles into wiki.

Relationship with Other Tools#

This skill doesn't replace any professional tool, it connects them:

text

Any research tool produces analysis → ingest into corresponding wiki
Any data tool pulls data           → ingest into corresponding wiki
Domain seeds provide starting line  → standard terms + anti-confusion rules
External validators check logic     → lint checks knowledge structure completeness

Next time executing tasks, agent reads relevant wiki → works with accumulated knowledge