TL;DR
Cortex, SAM, and SAME each passed their own benchmark. Then all three got the same deterministic event stream: Cortex held 99.3% accuracy at ~30 tokens per query, SAM returned a stale answer a third of the time, and SAME went silent on 71.7% of questions. The self-reported numbers predicted none of it.
The problem with self-benchmarks
Every memory system ships with a benchmark it wins. SAME reported 100% accuracy and zero emitted tokens on its effects path. Cortex reported 100% on its context-pack workload. SAM reported 97.7%. All three numbers are real, and none of them can be compared, because each system defines its own workload, its own token accounting, and its own notion of a correct answer.
The only way to get a comparable number is to stop letting each system pick the test. One event generator, one answer key, one definition of stale, one payload counter — then let the architectures differ.
What the contract looked like
A deterministic stream of durable decisions, explicit revisions of those decisions, non-durable status noise, and questions asked by a different agent than the one that wrote the memory. Then close every store, reopen it, and ask for the current state again. Scales of 250, 1,000, and 3,000 events across three seeds. No LLM calls anywhere in the loop, so nothing depends on a model having a good day.
An answer counts as correct only when the payload actually exposes the current value. Exposing a superseded value instead is stale. Exposing neither is missing. That three-way split matters more than accuracy alone, because stale and missing are different failure modes with different consequences.
correct = payload contains current value
stale = payload contains only a superseded value
missing = payload contains neither
payload_tokens = ceil(regex_words / 0.75)What it exposed
Cortex behaved the same at 3,000 events as at 250: 99.3% accuracy, 0.7% stale, nothing missing, payloads flat at ~30 tokens, p95 recall under 3ms, and 100% of final answers surviving a restart. Flatness across scale is the result here — the typed record plus explicit supersession does the work.
SAM degraded badly: a third of its answers were stale, and p95 recall went from 150ms to 4.2 seconds as the corpus grew. SAME never returned a single stale answer at any scale — it would rather say nothing than say something obsolete — but it said nothing 71.7% of the time and lost its runtime patch entirely across reload.
Keeping the result that hurts
Two of these three systems are ones we built to be interesting, and the shared contract says they are not ready. The honest reading is narrow: their adapters did not reproduce their native behavior under this contract. That is an integration bug to chase, not proof the ideas are wrong — SAME's separation of association from behavioral authority is still the right instinct, and it is exactly why it never went stale.
The wrong move would be to report the native table and quietly drop the shared one. The benchmark that makes your own work look worse is the only one that was actually measuring something.