What we learned building a temporal knowledge graph

The first version of Esment treated memory as a flat list: facts in, facts out, ranked by recency and importance. It worked, and it was wrong. Flat memory cannot answer the questions that matter — "how does this relate to that?" — and it cannot handle contradiction.

Why a graph

Facts are not independent. "Marc prefers dark mode" and "Marc works on Esment" are both true, but they are different kinds of truth: one is a preference attached to a person, the other an activity attached to a project. In a graph, memories become nodes grounded in entities, and relations become typed edges — USES, DEPENDS_ON, WORKS_ON, CREATES. Retrieval then does something a keyword search cannot: it expands. Ask about a project, and the graph walks out to the people, tools and decisions connected to it.

The lifecycle problem

A memory that cannot forget becomes noise. Every store needs a lifecycle: generated → activated → merged → archived → expired. The subtle part is merging — when new information contradicts old information, which one survives?

Our reconciler resolves temporal conflicts: a newer fact supersedes an older one, but the old fact is never deleted — it is archived, with the commit DAG preserving exactly what was known at every point in time. That turns the store into a time machine: what did we believe about this project in March? is a query, not a guess.

Retrieval is a cascade, not a search

We ended up with a four-stage pipeline because no single technique is enough:

1. FTS/BM25 — exact and keyword matches, millisecond-fast;
2. HNSW vectors — semantic similarity for things worded differently;
3. Graph BFS — expansion through relations, the stage that makes the graph pay for itself;
4. MMR reranking — diversity-aware final ordering so the injected context is not five near-duplicates of the same fact.

Each stage is fast alone; the cascade is what makes the whole thing feel like a memory rather than a search engine.

The lesson

Building a memory engine is 20% storage and 80% judgment: what to keep, what to merge, what to let die, and what to surface. The graph gave us the structure to make those judgments well. The lifecycle gave us the courage to make them at all.