Give your AI the knowledge it's missing.
Esoteric knowledge (EK) is knowledge not found in the weights of frontier LLMs — the tribal knowledge in your support team's heads, the gotchas your engineers learned the hard way, the decision patterns that were never documented. ExpertPacks deliver this knowledge to any AI agent in a way that minimizes token cost and maximizes prompt quality through RAG.
Every pack is measured by its EK ratio — the proportion of content that frontier models cannot correctly produce on their own. During hydration, every fact is triaged: esoteric knowledge gets maximum treatment, general knowledge gets compressed to scaffolding. The result is dense, high-value context that makes your AI genuinely expert — not just articulate.
🌐 expertpack.ai · 📦 Free Packs · 📖 Schemas · 🧪 Evaluation · 💎 Obsidian Compatible · 🗂️ Vault Template
ExpertPacks are valid Obsidian vaults — open any pack directory directly in Obsidian and get:
- Dataview queries — live tables filtering by content type, EK score, retrieval strategy, tags
- Graph view — visual map of how pack files relate to each other
- Tag pane — browse all content by type and domain tag
- Full-text search — across all pack content and frontmatter fields
- Templater — create new EP-schema-compliant files from templates
Every content file includes YAML frontmatter (title, type, tags, pack, retrieval_strategy). The repo includes a .obsidian/ reference config with Dataview pre-configured and example queries.
To use a pack in Obsidian:
- Copy the
.obsidian/folder from the repo root into your pack directory - Open that directory as a vault in Obsidian
- Install Dataview + Templater from Community Plugins
Starting a new pack from scratch? Use the Obsidian Vault Template — pre-configured folder structure, Templater templates for every content type, and a live Dataview dashboard.
Link format is standard relative Markdown — packs render correctly on GitHub, in any Markdown reader, and in Obsidian simultaneously.
Models don't know what they don't know. When a model confidently hallucinates, it doesn't trigger a search — it thinks it already knows. An ExpertPack loaded into context preempts the hallucination with the correct answer.
Search requires the right question. If the model doesn't know about a specific firmware bug, it won't search for the precise query that finds the fix. You can't search for knowledge you don't know exists.
Not all knowledge is on the internet. Source code analysis reveals undocumented behavior. Expert interviews capture tribal knowledge never written down. Person packs contain private stories and reasoning.
Capture a person — stories, beliefs, relationships, voice, and legacy.
Use cases: Personal AI assistant, family archive, memorial AI, digital legacy, founder knowledge capture
Deep knowledge about a product or platform — concepts, workflows, troubleshooting, and the edge cases documentation never covers.
Use cases: AI support agent, sales assistant, training tool, onboarding guide
Complex multi-phase processes — phases, decisions, checklists, gotchas, and the gap between the official process and what actually happens.
Use cases: Home building guide, business formation, project management, certification processes
Combine person, product, and process packs into a single agent deployment with role assignments, context control, and cross-pack conflict resolution.
Use cases: CEO AI assistant, multi-product support bot, company knowledge base
Open-source ExpertPacks built from real documentation, community forums, and source code analysis. Each pack shows its EK ratio — the percentage of content that frontier AI models cannot produce on their own.
| Pack | Type | Files | Size | EK Ratio | Description |
|---|---|---|---|---|---|
| Home Assistant | Composite | 132 | 600 KB | 54% | Smart home automation — protocols, automations, presence detection, ESPHome, voice, energy, community gotchas |
| Blender 3D | Product | 83 | 392 KB | 42% | 3D modeling, animation, sculpting, physics, rendering, Geometry Nodes, Python scripting, production workflows |
| Solar & Battery DIY | Composite | 53 | 291 KB | 52% | Residential solar — system design, panels, batteries, NEC code, permitting, installation, troubleshooting |
💡 These packs demonstrate the framework with substantive, practitioner-level content. Browse them to see what a well-built ExpertPack looks like.
- Point your AI at the schema — pick person, product, or process. The agent reads the schema and knows exactly what to build.
- Feed it knowledge — talk to the agent, point it at websites, drop in documents. It structures everything automatically, triaging each fact for EK during hydration.
- Deploy the pack — drop the folder into any AI agent's workspace. Instant domain expertise — no prompt engineering required.
- Measure & improve — run evals to measure EK ratio, correctness, and hallucination rate. Use results to guide targeted improvements.
| Platform | Integration |
|---|---|
| OpenClaw | Add pack path to memorySearch.extraPaths. RAG indexes all .md files automatically. |
| Cursor | Place pack in project. Cursor indexes workspace files for context. |
| Claude Code | Place pack in project. Reference from CLAUDE.md or let agent discover it. |
| Custom / API | Feed .md files into your vector store or context window. v4.1 concept atoms target 400–800 tokens with a 1,000-token hard ceiling, so files remain retrieval-ready without external chunking. |
| MCP Server | ExpertPack MCP serves any pack over the Model Context Protocol — connect Claude Desktop, Cursor, Windsurf, or any MCP host instantly. |
ExpertPacks go beyond basic RAG with a multi-layer retrieval system and EK-aware hydration.
Each concept is a single self-contained file carrying everything the retriever needs:
| Element | What It Does | Why It Matters |
|---|---|---|
| Opening paragraph | 1–3 sentences that define the concept in retriever-friendly terms | First chunk carries the core definition; no throat-clearing |
| Body sections | ##-delimited sub-topics |
Whole file travels as one retrieval unit; sections organize the narrative for the reader |
## Frequently Asked |
H3-per-question FAQ block | Natural-language query surface right where the concept lives |
## Related Terms |
Terms that don't stand alone | Co-locates glossary-style definitions with the parent concept |
## Related Concepts |
Wikilinks to siblings | Reader navigation and Obsidian graph view |
requires: frontmatter (v4.1) |
Directional dependencies on other atoms | EP MCP auto-includes required atoms in the retrieved context |
This replaces the v3.x pattern of separate summaries/, propositions/, per-domain glossary-*.md, and standalone faq/ directories — those aggregator files scored broadly on every query and displaced specific content at retrieval time. See schemas/rfcs/RFC-001-atomic-conceptual-chunks.md for the empirical findings that drove the change and schemas/references/granularity-guide.md for authoring decisions.
Every extracted fact passes through the EK triage pipeline:
- EK (model wrong/refuses) → Full treatment: dedicated concept atom with a retriever-anchored opening and detailed body (keep the whole atom ≤ 1,000 tokens; split into multiple atoms with
requires:if the concept genuinely spans more) - Partial (model vague) → Standard treatment, highlight the specific detail the model missed
- GK scaffolding (model correct, but needed for retrieval) → 1-3 sentences max, no dedicated file
- GK unnecessary (model correct, no EK depends on it) → Skip entirely
See the Hydration Guide for the full hydration pipeline.
Every pack can include an eval suite to measure quality. The eval schema defines:
- EK Ratio — proportion of propositions that frontier models can't answer without the pack
- Correctness — percentage of required facts present in responses
- Hallucination Rate — percentage of responses containing fabricated information
- Refusal Accuracy — percentage of out-of-scope questions correctly declined
- Retrieval Hit Rate — percentage of queries retrieving the expected source files
The included eval-ek.py tool measures EK ratio via blind probing across multiple frontier models.
The schema itself is now the chunking strategy. Author concept files as self-contained retrieval units (500–800 tokens, 1,000 token hard ceiling in schema v4.1). Any RAG chunker will pass these files through intact without splitting, preserving structure and metadata. Cross-atom dependencies are declared via the directional requires: frontmatter field and expanded at retrieval time by EP MCP.
Every content file can carry provenance frontmatter — id, content_hash, verified_at, verified_by — enabling auditable citations, change detection, and freshness tracking. The pack manifest includes a freshness block for sweep coverage metrics. See the Citation Response Contract for how retrieval systems should surface provenance. The micro-record exporter can now emit compact Agent Knowledge Schema (AKS) JSONL (--compact) with first-class id, canonical_path, source_span_uri, content_hash, source_checksum, verified_at, requires, and related fields for token-efficient agent pipelines.
Packs can generate a _graph.yaml adjacency file from wikilinks, related: frontmatter, and context hint comments. This enables GraphRAG traversal, structural analysis, and cross-pack linking. Generate with ep-graph-export.py.
ExpertPack MCP exposes any ExpertPack as expertise-as-a-service over the Model Context Protocol. Schema-aware hybrid retrieval (BM25 + vector), graph-aware traversal, frontmatter-aware indexing, provenance metadata extraction, and multi-pack routing — any MCP-compatible agent gets instant access to pack knowledge.
ExpertPack is an actively evolving framework. The table below shows which features are fully implemented, which are partial, and which are spec-level (defined but not yet wired end-to-end in the reference runtime).
| Feature | Status | Notes |
|---|---|---|
| Atomic-conceptual content model | ✅ Full | Schema v4.1; all new packs use this model |
requires: dependency frontmatter |
✅ Full | Spec + EP MCP runtime expansion live (depth 2, count 3 cap) |
| EK triage during hydration | ✅ Full | Spec + tooling (eval-ek.py) |
| Provenance frontmatter | ✅ Full | Spec defined; authoring + micro-record tooling honors it |
| Volatile data isolation | ✅ Full | volatile/ convention + frontmatter TTL |
| Automated eval runner | ✅ Full | eval-runner/ with LLM-as-judge scoring |
| Graph export | ✅ Full | ep-graph-export.py; _graph.yaml output incl. accepted ontology/entity edges |
| Agent Knowledge Schema (AKS) export | ✅ Full | ep-micro-record-export.py --compact; compact provenance-first JSONL for agent pipelines |
| Ontology suggestion CLI | ✅ Initial | ep-ontology-suggest.py; review-first entity/category suggestions plus accepted ontology.yaml registry |
| Pack validation CLI | ✅ Full | ep-validate --strict (22+ checks), ep-doctor, W-CHUNK sidecar rules; CI + pre-commit |
| Strict ingest gate | ✅ Full | tools/ingest-gate.py — validate → strip → export; no index without --strict |
| Fragment provenance / Reconstruct Mode | ✅ Full | RFC-003 in core; OpenClaw plugin sends reconstruct; EP MCP returns fragment_id / line_range / stale envelope |
| Chunk metadata sidecars | ✅ Full | RFC-004, ep-chunk-annotate.py, W-CHUNK-01..03; EP MCP consumes sidecars at index time |
| Typed Answer Contract (TAC) | ✅ Full | Spec + validate_tac.py + claim_verifier --tac + agent prompt template |
| MCP server (EP MCP) | ✅ Full | BM25 + vector hybrid retrieval; multi-pack routing |
| Provenance display in agent responses | ✅ Runtime-supported | EP MCP opt-in reconstruct mode; host renders fragment provenance |
| Scaffolding CLI / guided creation | ✅ Full | tools/cli/expertpack.py — init, validate, doctor, migrate, chunk-annotate |
| Process schema (v4.1) | ✅ Full | Migrated to atomic conceptual/procedural files, requires: phase dependencies, and core v4.1 size rules |
The canonical compatibility matrix lives in schemas/schema-index.yaml. Regenerate this table with python3 tools/update-schema-readme.py when schema versions change.
| Schema | Version | What It Covers |
|---|---|---|
| core.md | 4.1 | Shared principles: MD-canonical, atomic-conceptual model, requires:, provenance, Reconstruct Mode, chunk sidecars, TAC, --strict gate, graph export, registry projections |
| product.md | 4.1 | Product packs: concepts, workflows, interfaces, troubleshooting, commercial, customers |
| person.md | 4.1 | Person packs: stories, reflections, opinions, conversations, mind taxonomy, relationships, presentation |
| agent.md | 1.7 | Agent extension: persona, capabilities, tool access, behavioral rules |
| process.md | 4.1 | Process packs: phases, decisions, checklists, exceptions, scheduling, regulations |
| composite.md | 1.2 | Composites: multi-pack deployment, roles, fail-closed conflict isolation, auto-discovery & export |
| eval.md | 1.4 | Evaluation: EK ratio, correctness, hallucination, refusal/authority, composite conflict cases, TAC scoring |
| registry/agent-knowledge.spec.yaml | 1.0 | Compact Agent Knowledge Schema (AKS) JSONL — includes optional fragment provenance fields |
| registry/frontmatter.spec.yaml | 1.0 | Content-file frontmatter contract enforced by ep-validate --strict (companion JSON Schema) |
| registry/chunk-sidecar.spec.yaml | 1.0 | Chunk metadata sidecar format for oversized atomic/reference files (RFC-004) |
| registry/typed-answer.spec.yaml | 1.0 | Typed Answer Contract v1 — machine-verifiable claim-to-source agent responses |
| registry/ontology.spec.yaml | 1.0 | Accepted ontology registry descriptive spec for graph/entity interop |
ExpertPack development is guided by 13 axioms (1–10: EK and hydration; 11–13: retrieval-first authoring):
- Esoteric knowledge (EK) is knowledge outside the weights of frontier LLMs
- EPs have a subject matter — person, product, or process
- EPs maximize the ratio of EK to general knowledge
- Hydration is the process of populating the pack with EK
- Compaction minimizes loss of EK
- Quality = compactness × EK volume × retrieval quality × minimal decay
- Market value = quality × market potential × agentic correlation
- Cost increases with human exchange needed + compute tokens
- EK ratio is empirically measurable via blind probing
- Hydration should prioritize EK — minimize general knowledge, maximize what only the pack can provide
- One topic = one file — the retrieval unit is the file
- No standalone summary files — summaries belong in the lead sentence of the atom they describe
- Content strategy must match the retrieval model — retrieval-first vs LLM-reads-all
Operational contracts (strict gate, Reconstruct Mode, sidecars, TAC) implement axioms 6 and 13 — see AXIOMS.md § Operational contracts.
ExpertPack/
├── ARCHITECTURE.md ← Framework overview (how packs work)
├── AXIOMS.md ← 13 guiding axioms for EP development
├── CHANGELOG.md ← Framework changelog
├── ROADMAP.md ← Development roadmap
├── README.md ← This file
├── LICENSE ← Apache 2.0
│
├── schemas/ ← Pack blueprints (the framework)
│ ├── core.md ← Shared principles (v4.1)
│ ├── schema-index.yaml ← Compatibility matrix (source for README table)
│ ├── registry/ ← frontmatter, AKS, chunk-sidecar, TAC specs
│ ├── rfcs/ ← RFC-003 Reconstruct Mode, RFC-004 sidecars, …
│ ├── person.md ← Person-pack schema (v4.1)
│ ├── agent.md ← Agent subtype schema (v1.7)
│ ├── product.md ← Product-pack schema (v4.1)
│ ├── process.md ← Process-pack schema (v4.1)
│ ├── composite.md ← Composite schema (v1.2)
│ ├── eval.md ← Evaluation framework (v1.4)
│ └── references/ ← Extracted reference material
│
├── guides/ ← Practical guides
│ ├── hydration.md ← Hydration lifecycle: planning → population → retrieval optimization → validation
│ ├── consumption.md ← Deploying and consuming packs with AI agents (incl. deploy-prep and eval discipline)
│ └── intelligence-refinement.md ← Periodic framework research + schema review cycle
│
├── tools/ ← Tooling
│ ├── cli/expertpack.py ← Unified CLI (init, validate, doctor, migrate, …)
│ ├── validator/ ← ep-validate.py, ep-doctor.py
│ ├── chunker/ ← ep-chunk-annotate.py (RFC-004 sidecars)
│ ├── tac/ ← validate_tac.py (Typed Answer Contract)
│ ├── ingest-gate.py ← validate → strip → export pipeline
│ ├── validate-all.py ← CI/pre-commit runner
│ ├── check-schema-projections.py ← skill schema projection guard
│ ├── update-schema-readme.py ← README schema table from schema-index.yaml
│ ├── graph-export/ ← ep-graph-export.py — generate _graph.yaml adjacency files
│ ├── deploy-prep/ ← ep-strip-frontmatter.py — strip provenance metadata before RAG deploy
│ ├── micro-record-exporter/ ← ep-micro-record-export.py — full/compact provenance-first JSONL export
│ ├── ontology-suggest/ ← ep-ontology-suggest.py — review-first ontology/entity suggestions
│ ├── eval-runner/ ← Eval execution + claim_verifier (--tac)
│ └── eval-ek.py ← EK ratio measurement via blind probing
│
├── skills/ ← OpenClaw agent skills (also on ClawHub)
│ ├── expertpack/ ← Consume / hydrate / validate
│ ├── expertpack-eval/ ← EK ratio + quality evals
│ ├── expertpack-export/ ← Auto-discover & export agent → EP
│ └── expertpack-intelligence/ ← Framework research + schema review cycle
│
├── logs/intelligence/ ← Dated Intelligence Refinement run logs
│
├── template/ ← Obsidian vault template (DESIGN.md, TOOLS.md, …)
├── packs/ ← Community pack instances
│ ├── home-assistant/ ← Home automation (composite, EK 54%)
│ ├── blender-3d/ ← 3D software (product, EK 42%)
│ └── solar-diy/ ← Solar & battery DIY (composite, EK 52%)
└── site/ ← expertpack.ai website source
ExpertPack was designed and battle-tested with OpenClaw — the open-source AI agent platform. Every schema change is validated against real agent deployments.
Apache 2.0 — see LICENSE for details.
The ExpertPack framework (schemas, guides, tooling) is open source. Individual pack instances contain original content and can be licensed independently by their creators.
Website · GitHub · Built by Brian Hearn