Project File

SynapNet / HibbieFormer

Brain-inspired sequence models: O(L) recurrent cores, Hebbian plasticity phases, neuromodulation, and sleep-style consolidation — now converging on the HibbieFormer-HX program.

Active ResearchNot yet a validated breakthrough — and it says so on the blueprint.

What Was Actually Trained

MEASURED

The 2025 SynapNet Brain / HibbieFormer line trained recurrent bio-inspired models (gated state blocks + proto-neurons with 4 dendritic branches, 8 layers) at 13.5M, 27M, 60M, and 84M parameters, alongside a 163M transformer control. Training cycles explicitly alternate plasticity phases — hebbian, synaptic scaling, pruning — on a 50/30/20 schedule with an adaptive neuromodulator.

Documented Results

MEASURED

The completed TinyStories run logged 215 plasticity cycles over 21,476 steps, converging to PPL ~2.1–2.9, with a 64-prompt behavioral suite at cycle 150 confirming stable, domain-coherent generation. The strongest checkpoint result on record: the d512 recurrent core hit PPL 10.2 at step 5500 (down from 958 at step 500). SynapNet V3 (42.6M, MLX) measured PPL 66.03 on WikiText-2.

HibbieFormer-HX

HYPOTHESIS

The current blueprint specifies a falsifiable architecture program: a helical recurrent core (O(L), phase-encoded state), dendritic branch mixer with entmax competitive gating, surprise-gated fast hippocampal memory, slow cortical weights, an accelerated-replay sleep consolidator, and a neuromodulatory controller. Go/no-go gates are pre-registered — e.g. the recurrent core must beat HELIX throughput by ≥1.5× and sleep must cut forgetting by ≥20%, or the branch dies.

Forensic Honesty

OBSERVED

A read-only Stage-0 audit of the 2025 backup verified all 54 checkpoints intact (37.6GB) and flagged a real discrepancy: the file named 'best_verified' is byte-identical to the step-5000/PPL-12.2 checkpoint, not the PPL-10.2 one. That flag ships with the result.

HibbieFormer-HX Architecture Explorer

FROM THE HX BLUEPRINT — A RESEARCH PROGRAM, NOT A VALIDATED ARCHITECTURE

STAGE 01 — SENSORY BUFFER

Depthwise causal Conv1D (k=3–7) front-end — a thin sensory layer before the recurrent core.

Current Limitations

−HX is a research program, not a validated architecture — the blueprint states this in line 8.
−The 163M transformer control ran only 1,000 steps; cross-family comparisons are not yet fair.
−2025-era code is PyTorch/MPS; the fused Metal kernel work for HX starts greenfield.

Next Steps

+Stage 3: reproduce the HELIX baseline for a fair throughput gate.
+Implement the helical recurrent core on the mlx-recurrence kernel stack.
+Run the pre-registered go/no-go gates at 10–30M scale first.