kb-build: knowledge bases and specialist agents for Claude Code

Why I needed it

My agents work with a lot of rules: marketplace policies, how the ad platforms work, our own handbooks. When they answer from memory, the answer is often old or too general, and sometimes it's made up. The tricky part is that it sounds just as sure either way.

So every important topic gets its own knowledge base, with a specialist agent on top. Today I have ten of them, for things like ads, inventory, pricing and finance. Before an agent changes something in one of these areas, it asks the specialist first.

What matters most to me is that I can trust it. A knowledge base you can't test is a black box: you only notice that the search broke when an answer is wrong. So every knowledge base comes with a few test questions, and after every change a script checks that the right pages still come back.

kb-build is that system, without anything specific to my company. Claude Code wrote it, and Codex, GLM and Grok reviewed it, round after round, until Codex found nothing more. On the way, 34 real problems were fixed.

What it does

Collect

Name a topic, and Claude Code researches the top experts and official sources for it: several research agents at once, every address checked. You see the list and say yes before kb-build fetches the pages into your knowledge base, and your own files go in too. Before each fetch, and again at every redirect, kb-build checks that the address is a public https one. It has no tricks to get past bot protection: if a site turns it away, it accepts that and only tries again on a later run.

Index

The text is split along its headings and indexed on your own computer: the BGE-M3 model makes it searchable, and the Qdrant database stores it. Every topic gets its own collection, so topics never mix. After that, only what changed is indexed again, and kb-build refuses to empty a knowledge base by accident.

Test

Every knowledge base has three to five test questions, each with the passage it should find. After every change, a script checks that this passage is in the top five. It also asks off-topic questions, because in a small collection almost anything comes back, and a test that always passes measures nothing.

Who and when

Every piece of text keeps its source and, where known, its author and date, plus a trust level: your own notes rank high, official docs and curated experts medium, forums low. Search hits show how old they are when a date is known, and newer sources rank higher. When two sources disagree, the higher trust and the newer date win, and the specialist names both.

The specialist

A read-only agent for Claude Code. It searches the knowledge base before it says anything, names the source of every key point, and says "not in the knowledge base" instead of guessing. Anything from its own memory is marked as unverified.

Sources are data

Text in a page that reads like a typical order, for example "ignore your rules", gets flagged when it's indexed, and the specialist reports it instead of following it. The check is a list of phrases, so softer steering can slip through. That's why the specialist's own rules come first: whatever it reads is data, never an order. In the default setup, a guard also blocks every command except its own search.

Stays fresh

One scheduled job fetches older pages again, indexes them and reruns the tests for every active knowledge base. No AI agent is involved, only the local model that builds the index. You can mark outdated pages as replaced instead of deleting them, so the history stays citable. Finding new sources stays your call: the job reminds you when a topic is due for a fresh look. When something breaks, it raises an alert.

What it does on your computer

Your sources and the index stay on your computer, in folders that git ignores. After setup, indexing and search run on your computer too, including the index model, a one-time download of about 2.3 GB. Only two things go online: fetching web pages, and the specialist itself. It's a normal Claude Code agent, so what it reads goes to Claude like anything else in your session.

kb-build doesn't read robots.txt, the file where a site says what automated visitors may fetch, and it introduces itself as a normal browser, because many documentation sites turn away unknown clients. So check the terms of every site before you add it. What you fetch is for your own use, so don't publish it.

The command guard is only a backup, not a sealed box: it limits commands, but the specialist can still read any file your user account can read. Its own rules come first. All known limits are in the README.

Try it

You need Python 3.11 or later, Docker, about 2.3 GB for the model, macOS or Linux, and Claude Code 2.1.295 or later for the specialist's guard. Then:

git clone https://github.com/kevintheo-ai/kb-build && cd kb-build
python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python scripts/bge_embedder.py
docker compose up -d
cp config/kb_registry.example.yaml config/kb_registry.yaml
python scripts/kb_ingest.py demo
python scripts/kb_search.py --kb demo --eval

This builds the demo: the rules of a board game that doesn't exist. A model can't know them from memory, so it's easy to see whether an answer really comes from the knowledge base, and when the specialist says "not in the knowledge base".

To build your own, connect kb-build to Claude Code with ln -s "$PWD/skills/kb-build" ~/.claude/skills/kb-build and ask for a knowledge base, or type /kb-build. Claude Code asks what it's for and which sources to use, shows you the plan before it fetches anything, and only finishes when the test questions pass and the specialist has passed its checks.

kb-build has 142 automated tests, and they run without network, database or model.

Read the README on GitHub