LLM Wiki: A Structured Knowledge Management Pattern for Compounding Understanding

Intro to LLM Wiki

What Is It

LLM Wiki is a knowledge management pattern inspired by Andrej Karpathy’s approach. The idea is simple: instead of re-discovering knowledge from scratch every time you ask a question (which is what RAG does), you compile it once into a structured wiki and keep it current.

The result is a personal knowledge base that compounds over time. Every source you add strengthens existing pages, creates new connections, and fills gaps. The more you put in, the more valuable every page becomes.

The Problem It Solves

Traditional approaches have trade-offs:

• Bookmarks and folders – you save things but never find them again. No cross-referencing, no synthesis.
• Plain notes – better, but notes decay. You forget what you wrote, duplicate effort, and contradictions go unnoticed.
• RAG (Retrieval-Augmented Generation) – an AI searches your documents and answers from scratch each time. No memory of what it already figured out. No accumulated understanding. Ask the same question twice, it re-derives the answer twice.
• Zettelkasten / Obsidian – great for manual knowledge work, but the linking and maintenance burden is on you.

LLM Wiki splits the work: you direct the research and curate sources, the AI agent maintains the wiki – summarizing, cross-referencing, filing, flagging contradictions.

How It Works

The Three Layers

wiki/
├── SCHEMA.md           # Rules, conventions, tag taxonomy
├── index.md            # Content catalog with one-line summaries
├── log.md              # Chronological action log (append-only)
│
├── raw/                # Layer 1: Immutable sources
│   ├── articles/       # Web articles, blog posts
│   ├── papers/         # PDFs, arxiv papers
│   ├── transcripts/    # Interviews, meeting notes
│   └── assets/         # Images, diagrams
│
├── entities/           # Layer 2: People, orgs, products, models
├── concepts/           # Layer 2: Topics, ideas, techniques
├── comparisons/        # Layer 2: Side-by-side analyses
└── queries/            # Layer 2: Filed Q&A worth keeping

Layer 1 – Raw Sources. Immutable. The agent saves sources here but never modifies them. If you ingest a web article, the raw text lives in raw/articles/. Every raw file gets frontmatter with the source URL, ingestion date, and a SHA256 hash for detecting changes on re-ingest.

Layer 2 – Wiki Pages. Agent-owned. Each page is a markdown file with YAML frontmatter (title, dates, type, tags, sources, confidence level). Pages cross-reference each other with [[wikilinks]]. Minimum 2 outbound links per page – isolated pages are invisible in a wiki.

Layer 3 – Infrastructure. SCHEMA.md defines the domain, conventions, and allowed tags. index.md lists every wiki page with a one-line summary. log.md tracks every action chronologically.

Page Quality Signals

Every page carries metadata that makes the wiki self-auditing:

• confidence: high/medium/low – how well-supported are the claims? Single source? Multiple corroborations?
• contested: true – flags pages with unresolved contradictions
• contradictions: [page-name] – links to pages that conflict
• Provenance markers – on pages synthesizing 3+ sources, paragraphs end with ^[raw/articles/source.md] so you can trace each claim back to its origin

The Tag Taxonomy

Tags are constrained. SCHEMA.md defines 10-20 top-level tags for the domain. Every tag on a page must exist in the taxonomy. If a new tag is needed, you add it to the schema first. This prevents tag sprawl – the slow decay where “alignment”, “ai-alignment”, “AI safety”, and “safety-alignment” all coexist and nothing is findable.

Core Processes

1. Ingest – Adding Knowledge

You provide a source (URL, PDF, pasted text). The agent:

1. Captures the raw source – downloads the content, saves to raw/ with frontmatter (URL, date, SHA256 hash)
2. Discusses takeaways with you – what’s interesting, what matters for your domain
3. Checks what already exists – searches the index and existing pages to avoid duplicates
4. Writes or updates wiki pages – creates new entity/concept pages where warranted, adds new info to existing pages, bumps update dates, adds cross-references
5. Updates navigation – adds new pages to index.md, appends to log.md

One source can touch 5-15 pages. This is the compounding effect. A paper about transformer architecture doesn’t just create one page – it updates the transformer concept page, adds details to the attention mechanism page, links to the relevant researchers’ entity pages, and creates a comparison if it benchmarks against another architecture.

2. Query – Asking Questions

You ask a question. The agent:

1. Reads index.md to identify relevant pages
2. Reads those pages (and follows their links)
3. Synthesizes an answer citing the wiki pages it drew from
4. If the answer is substantial, files it as a query page for future reference

The key difference from RAG: the answer draws from already-synthesized, already-cross-referenced knowledge. The agent isn’t re-reading raw sources – it’s reading curated pages that already represent the best understanding of each topic.

3. Lint – Health Check

You ask to audit the wiki. The agent checks:

• Orphan pages – pages with no inbound links from other pages
• Broken wikilinks – [[links]] pointing to pages that don’t exist
• Missing frontmatter – pages without required fields
• Tag violations – tags not in the taxonomy
• Stale content – pages not updated in 90+ days despite new sources mentioning the same topic
• Contradictions – pages that conflict with each other
• Low-confidence pages – single-source claims marked as high confidence
• Page size – pages over 200 lines that should be split

Results are grouped by severity. Broken links and orphans are critical; style issues are minor.

Exemplary Usage

Scenario: AI/ML Research Wiki

Setup. You tell the agent: “My domain is AI/ML research.” It creates a schema with tags like model, architecture, training, inference, alignment, person, company, benchmark. The tag taxonomy constrains everything that follows.

Day 1: Ingest a paper. You share a link to the “Attention Is All You Need” paper.

The agent:

• Saves the PDF to raw/papers/attention-is-all-you-need-2017.md
• Creates entities/vaswani.md (lead author), entities/google-brain.md (org)
• Creates concepts/transformer-architecture.md with explanation, key innovations, and links to [[self-attention]], [[positional-encoding]], [[encoder-decoder]]
• Creates concepts/self-attention.md and concepts/positional-encoding.md as stubs with cross-links back
• Updates index.md with 5 new entries
• Logs everything in log.md

Day 3: Ingest a blog post. You share Karpathy’s “Let’s build GPT” tutorial.

The agent:

• Saves to raw/articles/karpathy-lets-build-gpt-2023.md
• Updates concepts/transformer-architecture.md – adds practical implementation details from the tutorial
• Creates concepts/byte-pair-encoding.md (tokenizer technique mentioned)
• Creates entities/openai.md (GPT’s org)
• Creates concepts/gpt.md linking to [[transformer-architecture]], [[byte-pair-encoding]]
• The transformer page now has 2 sources and gets bumped to confidence: medium

Day 7: Ask a question. “How does GPT differ from the original transformer?”

The agent:

• Reads index.md, finds concepts/gpt.md and concepts/transformer-architecture.md
• Reads both, follows links to concepts/self-attention.md
• Synthesizes: “The original transformer (Vaswani et al., 2017) is an encoder-decoder architecture for translation. GPT drops the encoder, uses only the decoder stack, and trains autoregressively on next-token prediction…”
• Files this as comparisons/gpt-vs-transformer.md – a durable comparison that will be updated as new sources come in

Day 14: Lint. You ask for a health check.

The agent finds:

• concepts/encoder-decoder.md is an orphan (no page links to it yet) – suggests adding links
• entities/google-brain.md has confidence: low with only one source – suggests finding corroboration
• No broken links, no tag violations

Scenario: Personal Health & Longevity Wiki

Same structure, different domain. Tags become supplement, biomarker, protocol, diet, study, mechanism. You ingest:

• A Peter Attia podcast transcript -> creates pages for concepts/zone-2-training.md, concepts/vo2max.md, entities/peter-attia.md
• A study on creatine and cognition -> updates concepts/creatine.md with cognitive benefits, links to concepts/cognitive-decline.md
• Your blood work results -> saved as raw, updates concepts/vitamin-d.md with your personal reference range

Six months in, you ask “what supplements have the strongest evidence for longevity?” The answer draws from 40+ curated pages, each already synthesized from multiple sources, each tagged with confidence levels. You get a ranked answer with provenance – not a fresh re-read of 40 raw sources.

Why It Works

1. Compounding. Every source strengthens existing knowledge. Page 100 is more valuable than page 1 because it links to 99 other pages.

2. Consistency. The schema and tag taxonomy prevent drift. Contradictions are flagged, not buried.

3. Navigability. index.md gives you the full picture in one page. [[wikilinks]] let you follow connections. The graph view in Obsidian shows the knowledge network visually.

4. Trust. Confidence levels and provenance markers tell you how much to trust each claim. Low-confidence single-source claims are surfaced for review, not treated as fact.

5. Durability. It’s just markdown files. No database, no lock-in. Works in Obsidian, VS Code, any editor. Syncs via git, Obsidian Sync, Nextcloud, or anything that handles files.

Obsidian Integration

The wiki is an Obsidian vault by default:

• [[wikilinks]] render as clickable links
• Graph View shows the knowledge network
• YAML frontmatter powers Dataview plugin queries: TABLE tags FROM "entities" WHERE contains(tags, "person")
• Set attachment folder to raw/assets/ for images

Existing Obsidian vaults can be wrapped – your original notes stay untouched, new wiki pages cross-reference them.

Based on Andrej Karpathy’s LLM Wiki pattern. Implemented as a Hermes Agent skill for agent-maintained knowledge bases.