The story of Knowl
Before summer I was working on a side project — nothing to do with any of this, just a thing I was building for myself. And the agent kept losing the plot.
Every new session started cold. I'd re-explain the same architecture, the same conventions, the same
three things it always got wrong. So I did what everybody does: I put it in CLAUDE.md. Then I put
the next thing in CLAUDE.md. And the next one.
By the time I looked properly the file was around a thousand lines, and here's the part that actually bothered me — I couldn't tell you which half of it was still true. Some of those lines described decisions I'd already reversed. Nothing in the file said which ones, because a markdown file has no idea when a line stopped being right, and neither the model nor I was going to re-derive a thousand lines to find out.
That eventually became Knowl: a local, structured memory store for coding agents that can retire a stale fact when newer knowledge supersedes it. Everything below is how I got there, including the approaches that didn't work.
So I went looking for a memory tool. There were plenty. Two things put me off all of them.
They were hard to install — a vector database to stand up, an API key to provision, a service to run alongside the thing I was actually trying to build. For a side project that is an absurd amount of setup to stop repeating yourself.
And the ones I did get running stored things well enough. What none of them convinced me they did was the thing I actually needed: decide that something I'd said before was no longer true.
There was also a less noble reason, which is that I didn't want to wait for anybody. Every time I wanted something small — a different retrieval default, a command that drew me the graph, a category that matched how I actually think about my own notes — the move was to open an issue and hope it lands two releases from now. Which is a perfectly reasonable thing to do about someone else's project.
So I did the proportionate thing and wrote an entire memory engine to avoid waiting on a flag, lol.
In fairness it has worked out, in the sense that the quality-of-life features now arrive on the Tuesday I want them, and in exchange all I had to do was own a database.
Everyone has supersession. That is not the question.
I should be straight about this, because it is the first objection anyone who has used one of these tools will raise. They mostly do have supersession. mem0 ships it, Zep ships it, agentmemory ships it. If your objection to this post is "that's a solved problem", you're right about the feature list.
So rather than tell you a story about a tool breaking on me, here are two things I measured, both of which you can check. Versions are pinned, because "it does X" about a moving repository is not a claim anybody can verify or refute:
The similarity can run backwards. agentmemory — read at tag v0.9.29, commit 2d38daf, on
2026-08-18, in src/functions/remember.ts — supersedes when Jaccard similarity over the whole
content exceeds 0.7. Take a pair that cannot both be true:
goaltender is associated with the sport of ice hockey
goaltender is associated with the sport of pesapallo
Shared tokens 7, union 10, Jaccard exactly 0.7000 — and the comparison is a strict >, so it
falls through. Checked against a running server: both records came back isLatest: true, neither
recorded a supersession, and the stale one ranked first.
The boundary is not the interesting part. This is: Jaccard over whole content is length-sensitive, and for this job that sensitivity runs the wrong way — changing more of a fact can lower the similarity even when the new fact is a direct replacement. Worked through in full here.
And catching it isn't the end of the job. Graphiti, under Zep, genuinely computes valid_at and
invalid_at on ingest. Then look at the read path. In graphiti_core/search/search_filters.py both
temporal filters are declared Field(default=None); across the two thousand lines of
search_utils.py, expired_at appears only as a projected output column and never once in a
WHERE clause; and mcp_server/src/graphiti_mcp_server.py constructs
SearchFilters(node_labels=...) with no temporal argument at all.
Read at commit 683a853, re-checked 2026-08-26. If that changes, this paragraph is wrong and I'd
like to know — it's four greps.
The expiry is computed, stored, handed back as a field, and left for the model to notice.
So the question was never "does this system have supersession." It is whether the check fires when the words don't line up, and whether the read path acts on it when it does. Those are the two things the rest of this post is about.
The first thing I got wrong
I assumed this was a retrieval quality problem. Better embeddings, a reranker, more careful chunking — surely one of those fixes it.
It doesn't, and understanding why is the whole design.
Look at what a stored fact and its correction actually look like. They are about the same subject, in nearly the same words, at nearly the same length — one value swapped. They are maximally similar to each other. That isn't a phrasing accident; it's precisely what makes them a contradiction instead of two unrelated facts.
So any retriever good enough to find one will find the other. Similarity has no opinion about which of two contradictory statements is current, because currency isn't a property of the text. It's a property of the history — and history is exactly what an append-only store throws away.
That reframed the problem. It isn't "retrieve better." It's "decide, at some point, that one of these is dead."
Where that decision can live
Three options, and I worked through all of them.
At read time, you rank and hope. Ruled out by the argument above — the two candidates are the most similar pair in the store.
In the model, you instruct it to notice contradictions — Letta does this in a reflection prompt, and I tried it too. It degrades silently under exactly the conditions where it matters: long context, many candidates, and a contradiction split across two chunks the model never sees together. It works in the demo and stops working at scale, which is the worst failure profile there is.
At write time, you have both records in hand at the moment one arrives. It's the only point in the lifecycle where the comparison is cheap, total, and guaranteed to happen.
Write time it is. Then came the part that took months, because knowing where to put the decision tells you nothing about how to make it.
Failure 1: the similarity model
The obvious implementation. Embed both atoms, threshold the cosine, call anything above the line a contradiction. I built the measurement harness for it first.
On the labelled pair set:
| pair | should | cosine |
|---|---|---|
| a real reversal | be caught | 0.8593 |
| hand-labelled negative | not be caught | 0.8575 |
Eighteen ten-thousandths apart. No threshold exists that separates them.
I assumed raw cosine was just the wrong readout, so I tried five derived statistics on the same set: margin, ratio, margin-over-standard-deviation, z-score, and an exponential tail. All five discriminated worse than the raw cosine.
The model isn't bad. It's answering a different question. "These two texts are about the same thing" and "this text declares that thing over" are orthogonal, and a reversal is maximally similar to what it reverses.
There's a second reason I'd have abandoned it anyway: similarity gates need a distribution to calibrate against. On a fresh store with two atoms there is no distribution, so the guard abstains by construction — precisely when a new user is deciding whether the tool does anything at all.
Failure 2: the subset test that fired backwards
What replaced it is lexical. sameSubjectTitle tokenises titles on [^a-z0-9_]+, drops stop words,
and if one title's token set is a subset of the other's, they're about the same subject and the
newer retires the older.
A subset rather than a threshold, because thresholds need per-store calibration and subsets don't. "Rate limit on the public API" and "Rate limit on the public API is 500/min" stand in that relation on a two-atom store and a ten-thousand-atom store alike.
Then my own store ate one of my decisions.
None of the words that flip a claim are stop words. Which means an affirmative title is a strict subset of its own negation. Running the real function:
"Push gate blocks default branch" SUPERSEDES "Push gate no longer blocks default branch"
"Reranker is the right call" SUPERSEDES "Reranker is not the right call"
The survivor asserts the opposite of what it retired. Silently. That first pair is this project's own push-gate reversal — a title shape that, written twice across a reversal, retires its own predecessor in whichever order the two writes happen to land.
The fix is a polarity guard: not, no, never, none, nor, without, cannot, longer,
unable, plus the bare contraction stems isn, aren, doesn, didn, hasn, couldn, wouldn.
Differ by one of those and the subset relation isn't a supersession.
Deliberately narrow — can is excluded, because "Can the gate block the branch" is a real title. The
asymmetry is on purpose: a missed negation costs what today already costs, while a false positive
costs a supersede that should have happened and now has to be made by hand.
Failure 3: cues that fired on everything
Titles don't catch everything. This pair, reproduced through the MCP path a real agent takes:
"Database choice: Postgres for everything"
"We are moving persistence to SQLite"
content: "The Postgres-for-everything plan is abandoned."
No subset relation between the titles. Whole-text overlap 0.33, just under the 0.35 gate. The
write lands in silence — both decisions active, both returned fresh, nothing for knowl conflicts
to show.
What separates the pair isn't closeness. It's that one of them says the other is over. Lexical again, so: a list of reversal phrases.
My first list was much longer. "instead of", "dropped", "rejected", "retire" — all obvious, all wrong. Run against a real 831-atom store they fire constantly, because that is just how engineers write. "We dropped the retry" isn't a reversal of anything; it's a sentence.
What survived is short: no longer, abandoned, superseded, deprecated, reversed, obsolete,
replaced by, overturned, rescinded, retracted. Even those fired on 148 atoms in that store,
and 129 of 1,033 in this one. I state that rate rather than tuning it away — it's a cue for a closer
look, not an automatic retirement.
Failure 4: the guard that was obviously right
This is the one I think about most, because I would have shipped it.
Atoms carry provenance: observed (something ran and this was the result), user_stated (a human
said so), inferred (concluded without direct evidence). So here's a rule that sounds
unimpeachable — inferred evidence must not overturn an explicit fact. If the held atom claims
observed and the incoming one doesn't, don't supersede. Let them coexist.
Before merging it I replayed it against every real supersession in the store. 101 of them.
| held → new provenance | count |
|---|---|
| observed → observed | 49 |
| null → unset | 41 |
| null → observed | 7 |
| observed → unset | 3 |
| null → user_stated | 1 |
The guard blocks 3 of 101. Three percent — small enough that I'd have called it acceptable and moved on.
Then I read the three. All of them are legitimate corrections. A 100% error rate inside the blast radius:
- "A helper reachable from a tool call must swap databases with withDbPath" → "Only withDbPath is safe inside an MCP request" — the same fact, sharpened.
- "Review blockers at 626799c: consent, ledger state…" → "5.0 spec revised: all eight review findings resolved" — a state update. Blocking it leaves resolved blockers asserted as open.
- A card-size measurement, corrected by a later measurement.
And "coexist is safe, nothing is lost" turns out to be false at the store level. A coexisting
contradiction is invisible afterwards: knowl conflicts only reads atoms carrying a conflictKey,
which is set on 3 of 937 active atoms. The near-duplicate advisory fires once, at write time, to
whoever reads the write result — and when that's an automated hook, there is nobody there to act on
it.
So the guard was removed before merge, with the measurement written into a code comment and pinned by a test, so the idea can't come back without new evidence. It's the clearest lesson I got out of building this: a rule that sounds obviously correct and has a 3% blast radius still has to be checked against the actual 3%.
Failure 5: capture, which is still not solved
None of the above matters if nothing gets written down.
The first design leaned on automatic promotion — let the session finalizer notice what mattered and store it. I measured what it actually produced across 467 memory sessions: 371 skipped, 61 pending, 36 promoted. Forty atoms lifetime, against 893 in the store. 4.5%.
Then I read all forty titles. They are exactly two shapes: git commit subjects, and "Resolved failure in X" entries. Not one is a finding, a comparison, or a decision. The extractors could recognise a commit and a failure/fix pair, and had no detector for "a decisive result" at all.
I've since eliminated two more approaches. Per-topic nudging fires on a third of already-disarmed sessions — the precision trap the static-analysis literature puts at 18–86% false positives.
This is the honest state of it: the write path is the part I trust; the capture path is only partially solved. Hooks catch a lot, the explicit tools catch the rest, and the fully-automatic version isn't there yet. The remaining shape I believe in is judging the session at finalize time with a model, offline, gated on its own calibration — which is a real design, expensive, and not built.
What finally worked
Knowledge is split into atoms rather than one growing document, each typed as fact, decision,
goal, constraint, architecture, state or skill, each carrying provenance, evidence, the
repository paths it depends on, and its supersede lineage.
On write, the lexical checks above run in code — inside knowl_store, on every write, whether or not
the agent thought to ask. An agent can pass an explicit supersedes: <id>; that isn't the path
doing the work.
Here is the whole idea in four commands, pasted from a throwaway repo rather than typed into this post:
$ knowl store "The public API allows 100 requests per minute per key" \
--title "Rate limit on the public API" --category fact
Stored fact 9774ecc8a8124836: Rate limit on the public API
$ knowl store "The public API allows 500 requests per minute per key" \
--title "Rate limit on the public API is 500 a minute" --category fact
Stored fact 36a3e798f3394e91: Rate limit on the public API is 500 a minute
Retired 9774ecc8a8124836.
$ knowl list
ID CATEGORY AGE READS TITLE
36a3e798 fact 0d 0 Rate limit on the public API is 500 a minute
$ knowl timeline 9774ecc8a8124836
"content": "The public API allows 100 requests per minute per key",
"validFrom": "2026-08-27T09:11:04.798Z",
Nobody asked for the supersession. The second title is a superset of the first, so the write path
retired the old fact on its own and said so. list shows one row because the retired one has left
the default view — and timeline still has it, with the window it was true for.
That is the entire product. Everything else is detail.
On read, knowl_query defaults to status: active. Retired atoms leave the default result set — you
ask for them by name. That one default is most of the practical difference between this and a store
that merely records staleness.
And because atoms record which files they depend on, a result can tell you the ground moved underneath it:
2 of 2 affectedPaths modified since this was stored -- verify against the files before trusting
Nothing contradicted that fact. Nobody superseded it. The code just changed, and the atom says so instead of quietly being wrong.
Nothing is ever deleted. Supersession sets superseded_by_id and leaves the row; there is no delete
verb in the product. The store stays bounded through a GC pass that moves cold atoms to archived,
and knowl gc --preview shows what would move before anything does.
Does it work
MemoryAgentBench has a task built for exactly this — FactConsolidation stores a fact, changes it, and asks which holds. Single-hop at 262K context:
The dotted line is the best published figure in the table — GPT-4o long-context at 60%. The two bars above it, Knowl and agentmemory, are my own runs on the benchmark's own harness; everything else is from the MemoryAgentBench paper.
| system | SubEM |
|---|---|
| Knowl | 0.90 |
| agentmemory | 0.79 |
| GPT-4o (full context) | 0.60 |
| HippoRAG-v2 | 0.54 |
| BM25 | 0.48 |
| Cognee | 0.28 |
| MemGPT | 0.28 |
| Mem0 | 0.18 |
| Zep | 0.07 |
I ran Knowl and agentmemory myself on the benchmark's own harness at temperature 0.7. The rest are the published MemoryAgentBench figures (arXiv 2507.05257v4, Table 3, gpt-4o-mini reader) — different runner, different day, and I'd rather say so than bury it. A cross-vendor table always has that weakness, so here is the same claim without it.
The ablation. Same code, same corpus, same 100 questions, one flag:
| supersession | top-1 | stale answers returned |
|---|---|---|
| on | 98% | 2 |
| off | 47% | 62 |
That is Knowl against Knowl. Nobody else's numbers are involved, so there is nothing to argue about except whether you believe my harness — and you can run it:
npm run bench:cr -- run --instance benchmarks/memoryagentbench/data/cr-sh-6k.json
Every recorded run is committed as raw JSON in benchmarks/memoryagentbench/results/, with a README
that says which fields are comparable and which runs are historical. Two of them are marked not
comparable to anything above because they predate a change to the embedding preset — I left them
in, labelled, rather than quietly deleting the runs that muddy the story.
On multi-hop I score 0.07, against an all-time ceiling across every system of 0.14. Chaining two supersessions is unsolved, by me and by everyone. Full results and the splits where Knowl does worse are in the repo.
Getting it running
The install complaint from the top of this post was the one thing I wasn't going to reproduce. Node 20, one command, no API key, no vector database, no service:
npm install -g @dat999zx/knowl
knowl init
knowl init writes knowl.db at your repository root. That file is the entire store.
Connecting an agent is one command each — and everything speaks MCP, so anything not listed works anyway:
knowl init claude # Claude Code MCP + hooks
knowl init codex # Codex CLI MCP + hooks
knowl init cursor # Cursor MCP + hooks
knowl init copilot # GitHub Copilot MCP + hooks
knowl init antigravity # Antigravity MCP + hooks
knowl init windsurf # Windsurf MCP + gate
knowl init cline # Cline MCP + plugin
knowl init opencode # OpenCode MCP
knowl init claude-desktop # Claude Desktop MCP
knowl acp # Zed, JetBrains, Neovim, Kiro
Start with one marked hooks. MCP alone gives the agent tools it may choose to call; hooks are what let it capture without being asked, which — given Failure 5 above — is the difference between a memory you have to remember to use and one that fills up on its own.
Don't migrate your CLAUDE.md. I tried that first. You import a thousand undated claims, several
already false, and now they're atoms with no provenance and no evidence. Connect one repository and
let a week of ordinary work accumulate instead.
Then:
knowl view # the graph, categories, supersede lineage
knowl conflicts # pairs that could not be adjudicated automatically
knowl gc --preview # what would be archived, before it is
knowl view is when it clicked for me — seeing the supersede edges drawn, this fact replacing that
one on this date because of this run, is when it stopped being a database and started being a record.
Everything above is local and free. knowl.cloud is the optional layer for sharing a workspace
across a team; queries never reach the server, so reading is free at every tier.
Apache-2.0, all of it: github.com/dat999zx/knowl
What I'm still unsure about
Retirement happens automatically, at write time, with no confirmation step.
I chose that because a prompt nobody answers is just a slower way to go stale — if you approve every supersession, you'll read the first ten and click through the rest, which is worse than automatic because now it has your name on it.
But Failure 2 is exactly the class of thing automatic retirement gets wrong quietly, and I only caught it because it ate a decision of mine that I happened to remember making. If you hit a case where it goes wrong, I'd genuinely like to hear about it.
The longer technical version of the similarity measurement is here.