Skip to content
D-CSIL

Experiment Record · 2026-07-22

MEM-SHARED-3K

One contract, three memory systems: Cortex vs SAM vs SAME

SUCCESSMEASURED

Hypothesis

Three local persistent-memory architectures can only be ranked fairly under a single deterministic contract — one event generator, one answer key, one stale definition, one payload counter.

Setup

A deterministic stream of durable project decisions, explicit revisions, non-durable status noise, and questions issued by a different agent than the writer, replayed into all three adapters at 250 / 1,000 / 3,000 events across seeds 0, 1, and 2. Each system used its native transport: Cortex typed memory records, SAM MTROI-selected reconstruction text, SAME structured runtime patch. Persistence was tested by closing and reopening each isolated store. Zero LLM calls, zero mutations to live production memory, temporary stores only.

Variables

  • Memory architecture (Cortex / SAM / SAME)
  • Event-stream scale
  • Random seed

Results

  • Cortex at 3,000 events: 99.3% current-answer accuracy, 0.7% stale, 0.0% missing, 30.3 normalized payload tokens/query, 2.936ms p95 recall, 100% restart accuracy.
  • SAM at 3,000 events: 45.5% accuracy, 36.2% stale, 18.2% missing, 219.1 tokens/query, 4,213ms p95 recall, 49.3% restart accuracy.
  • SAME at 3,000 events: 28.3% accuracy, 0.0% stale, 71.7% missing, 24.0 tokens/query, 8.134ms p95 recall, 0% restart accuracy.
  • All three passed their native regression suites (SAM: 34 passed, 1 skipped).

Observations

  • ·Native benchmarks flattered everyone — SAME 100% accuracy, Cortex 100%, SAM 97.7% — but each measures a different token model and workload, so none of them ranks anything.
  • ·SAME never returned a stale value at any scale. It withholds rather than asserting an obsolete fact — a genuine safety property, entirely separate from its recall failure.
  • ·SAM's recall p95 degraded from 150ms at 250 events to 4.2 seconds at 3,000: a scaling problem, not a tuning problem.
  • ·The shared harness itself cost 2,889s (~48 minutes). Reporting evaluation overhead is part of an honest benchmark.

Conclusion

Cortex is the deployment baseline for shared lab memory: stable across scale, ~30-token payloads, no missing answers, and full restart persistence. SAM and SAME stay research systems — their shared-contract adapters did not reproduce their native performance, and that gap is an open investigation, not a disproof of the underlying ideas.

Next Experiment

Diagnose SAME's post-reload runtime-patch loss and SAM's formation/supersession behavior against the same corpus, then require a measured improvement before either enters the critical memory path.