llmwiki#

A personal knowledge base that an LLM compiles, not just stores. Raw documents go in, an LLM writes trilingual (EN / 中文 / 日本語) wiki articles with [[wiki-links]], backlinks, and an emergent taxonomy. The MCP server dispatches every tool through llmwiki/operations.py; the CLI exposes the same registry via llmbase ops call; individual HTTP/CLI wrappers are being migrated onto the registry over time.

• PyPI: pip install llmwiki
• CLI command: llmbase (the package name and the command differ)
• GitHub: https://github.com/Hosuke/llmbase
• Demo: https://huazangge-production.up.railway.app

Setup#

bash

pip install llmwiki

mkdir my-kb && cd my-kb

cat > .env << 'EOF'
LLMBASE_API_KEY=sk-your-key
LLMBASE_BASE_URL=https://your-endpoint/v1
LLMBASE_MODEL=your-model
# Optional: LLMBASE_FALLBACK_MODELS=backup-1,backup-2
EOF

cat > config.yaml << 'EOF'
llm:
  max_tokens: 16384
paths:
  raw: "./raw"
  wiki: "./wiki"
EOF

Commands#

CommandDescriptionllmbase ingest url <url>Ingest a web articlellmbase ingest pdf <file>Ingest a PDF (auto-chunks)llmbase ingest file <file>Ingest any local filellmbase ingest dir <dir>Ingest all files from a directoryllmbase ingest cbeta-learn --batch 10Corpus plugin: Buddhist canonllmbase ingest ctext-book 论语 /analects/zhCorpus plugin: Chinese classicsllmbase compile newCompile new raw docs incrementally (3-layer dedup)llmbase compile allFull rebuildllmbase compile indexRebuild index + aliasesllmbase query "<q>"Ask a question (single-pass; add --deep for multi-step research)llmbase query "<q>" --tone wenyan📜 classical Chinese voicellmbase query "<q>" --tone scholar🎓 academic voicellmbase query "<q>" --tone eli5👶 simple voicellmbase query "<q>" --tone caveman🦴 primitive voicellmbase query "<q>" --file-backFile answer back into the wikillmbase lint check8-category structural health checkllmbase lint healCheck → fix → re-check → reportllmbase lint deepLLM deep quality analysisllmbase webWeb UI at :5555llmbase serveAgent HTTP API at :5556llmbase mcpStart MCP server (stdio)llmbase statsKB statistics

MCP Integration (for AI clients)#

json

{
  "mcpServers": {
    "llmwiki": {
      "command": "python",
      "args": ["-m", "llmwiki", "--base-dir", "/path/to/my-kb"]
    }
  }
}

Tools exposed by the MCP server:

ToolPurposekb_searchFull-text search over compiled conceptskb_search_rawVerbatim full-text fallback over raw/ sources (v0.6.2+)kb_askDeep-research Q&A with tone modeskb_getGet article by slug or alias ( 空, kong, emptiness all work)kb_listList articles, filter by tagkb_backlinksFind articles citing a given articlekb_taxonomyMultilingual category treekb_statsArticle count, word countkb_xiciGuided reading (导读)kb_ingestIngest a URLkb_compileCompile raw → wikikb_lintHealth check / auto-fixkb_export / kb_export_article / kb_export_tag / kb_export_graphStructured export for downstream projects

All tools are declared in llmwiki/operations.py — downstream projects register custom ops via operations.register(...) and they become available on CLI + MCP automatically.

Agents mounted on this server can answer from compiled concepts, fall back to raw sources with kb_search_raw when compile glossed a detail, ingest new material mid-session, and trigger healing.

Workflows#

Build a KB from scratch#

text

llmbase ingest url https://example.com/topic
llmbase ingest pdf ./paper.pdf
llmbase compile new
llmbase query "What are the key concepts?"
llmbase lint heal

Autonomous mode (deploy once, server keeps learning)#

yaml

# config.yaml
worker:
  enabled: true
  learn_source: cbeta         # built-in: cbeta | wikisource | both; custom via register_learn_source()
  learn_interval_hours: 6
  compile_interval_hours: 1
  health_check_interval_hours: 24

health:
  auto_fix_broken_links: true
  max_stubs_per_run: 10

The worker starts under the production WSGI entrypoint ( wsgi.py → start_worker_thread). Deploy with gunicorn wsgi:app; llmbase web alone does not self-start the worker.

Daily use as agent memory#

1. Agent receives a task → calls kb_search for relevant concepts
2. If the compiled answer is too abstract → calls kb_search_raw for verbatim detail
3. Learns something new → calls kb_ingest with the URL
4. Optionally kb_compile to fold it into concepts for next session
5. Periodically kb_lint heals the graph

Key Concepts#

• Synthesis, not archiving — LLM reads raw material and writes composed articles; storage is the cheap part
• Two-layer recall — kb_search (concepts) + kb_search_raw (verbatim raw sources)
• Trilingual default — every article has EN / 中文 / 日本語 sections
• 叠加进化 — new data merges into existing concepts, never overwrites
• Domain-agnostic — taxonomy emerges per-domain, nothing hardcoded
• Self-healing — 7-step auto-fix pipeline repairs drift
• Alias resolution — [[参禅]] → can-chan.md across scripts and simplified/traditional
• Registry-backed ops — MCP dispatches every tool through operations.py; CLI exposes the same registry via llmbase ops list / llmbase ops call; direct HTTP/CLI wrappers are being migrated onto the registry

Tips#

• --file-back saves Q&A answers into the wiki so future queries benefit
• --tone wenyan for Chinese users (classical Chinese responses)
• Run llmbase lint heal after large ingestion batches
• Web UI /health has buttons for every repair op
• Knowledge graph at /graph — density slider for large KBs
• Timeline at /explore — requires entities: { enabled: true } in config

Security & Privacy#

• All data stays local — wiki files are plain markdown on your filesystem
• LLM API key — user-supplied, loaded from .env
• Network access — user-initiated (URL ingest, SSRF-protected) plus corpus plugins ( cbeta-learn, wikisource-learn, ctext-book) and the autonomous worker when enabled
• Web server — optional; binds 0.0.0.0 so LAN-accessible by default — front with a reverse proxy or bind override for public exposure
• API secret — cloud deployments (with PORT env) gate most mutating endpoints behind LLMBASE_API_SECRET (auto-generated if unset). Note: /api/ask is open by default and writes Q&A back via file_back; only promotion to concepts requires the secret
• Autonomous worker — opt-in via config, disabled by default
• No telemetry — nothing is sent anywhere except the configured LLM API