Model Registry · HELIX family
HELIX-120M
H-E-R stack · HER×5 (15 layers) · d_model 768 · 12 heads · rotation-HSL (r_max 96) + ElasticAttentionGate + ResonantExpertRouting (MoE top-2 of 4)
Benchmarks
| Metric | Value | Notes |
|---|---|---|
| Held-out val PPL @ step 152,588 | 39.83 | corrected number — the in-loop '30.07' was measured val_path==train_path and is invalid |
| Matched transformer control (same data/steps) | 33.86 held-out | transformer leads by 5.96 PPL = 15.0%, matched protocol |
| Open-SLM-style benchmark, best checkpoint | 38.27% | identity-clean-sft-v1 @ step 168,823 — best of any 120M checkpoint |
| 5B-token extension | aborted ×3 | best held 37.84 @161k then degraded to 39.68 @187k — parameter-bound verdict |
Known Limitations
- −The headline foundation val PPL was originally reported as 30.07 — that number is invalid (val_path==train_path, an in-sample measurement); 39.83 is the corrected held-out figure and the one that should ever be cited.
- −A 5B-token continuation was attempted three times; the first two carried an absolute-step warmup bug, and the corrected run still saturated and degraded, indicating the architecture at 127M is parameter-bound, not schedule-bound.
- −Loss-curve anchor points are sparse (only start/end steps evidenced); the true training trajectory is not logged at fine granularity in the registry.
- −Not publicly released — served through a custom, Apple Silicon-only inference engine behind an invite-only gate. There is no public checkpoint, API, or download.