HELIX

HELIX is the lab's flagship model family. Each model stacks a repeating H-E-R pattern — HarmonicStateLayer (recurrent state), ElasticAttentionGate (sparse attention), ResonantExpertRouting (top-2 of 4 mixture-of-experts) — over a GPT-2 BPE vocabulary of 50,257 tokens, native to MLX on Apple Silicon with custom Metal kernels for the recurrent layers.

Three generations have trained: HELIX-56M and HELIX-120M (57.26M / 127.25M params, both complete) run today as a private, invite-only chat service at helix.derringtoncollaborativeai.com — a custom, Apple Silicon-only inference engine, no public checkpoints or API. HELIX-200M (209.05M params) has a complete base pretrain and active post-training. 127M and 209M were each trained from scratch — no weight inheritance between generations. What carried forward was architecture, code, recipes, identity data, eval machinery, and lessons.

HELIX-200M's base pretraining is complete: 244,141/244,141 steps, finished 2026-07-17, final held-out validation PPL 27.50 (28.45 on the trainer's own val split). Post-training reached a selected checkpoint — DPO step 100, chosen 2026-07-24 (behavior score 0.576, identity 0.667) — but a further SFT round is in progress now. HELIX is active research, not a finished artifact; every number on this page is subject to revision as that work continues.

Every HELIX generation has been checked against a matched dense transformer trained on identical data, steps, and protocol — and the transformer has won on perplexity every time it's been measured. At 56M: HELIX 27.29 tok / 48.59 word test PPL vs. transformer 25.10 / 44.82 — a 0.084-nat gap, roughly 8–9%. At 127M, held-out: HELIX 39.83 vs. transformer 33.86 — a 15.0% gap. The tax has never been re-measured at 209M scale, which makes it the single open question that decides whether HELIX's O(L)-inference design is worth its cost as parameters grow.

The program's best methodological result came from two plateaus that looked identical and had opposite causes. At 127M, a 5B-token continuation attempt saturated and degraded under a correct training schedule — the model was parameter-bound, and the fix was to scale up (the direct motivation for building HELIX-200M from scratch). At 209M, the foundation run plateaued at ~39–40 validation PPL under a schedule that was accidentally never decaying its learning rate. A recovery run — optimizer reset, proper re-warmup, then a real WSD decay — bought roughly 9 PPL from the decay phase alone (37.32 → 28.45), with zero architecture changes. Same symptom, two generations, opposite prescriptions, both confirmed by controlled before/after comparisons.

The founding architecture hypothesis — an absolute-time-gated 'pulse' recurrence — was falsified by the lab's own controlled bakeoff on 2026-06-11 and retired before any generation was trained on it. The 127M foundation's first validation number (30.07) was later found to be measured against its own training data (val_path==train_path); the corrected held-out figure is 39.83, and 'true held-out validation for every run' became a standing rule afterward. Four separate attempts at direct supervised fine-tuning on the 209M base all failed — identity collapse, zero improvement over the starting checkpoint, or abandonment mid-run — before the team fell back to mirroring the two-stage recipe that had worked at 127M. An entire 'v2' optimization layer was written, never passed its own correctness suite, and was quietly never used in production.