Skip to content
D-CSIL

Experiment Record · 2026-06-10

MLXR-HOTSWAP

mlx-recurrence v2 kernels hot-swapped into a live training run

SUCCESSREPLICATED

Hypothesis

Fused Metal scan kernels can replace the chunked MLX path mid-flight in a live SSM+GLA training run with no loss discontinuity and material memory/throughput gains.

Setup

Production D-CSIL SSM+GLA hybrid training run; swap chunked ops for ssd_scan/gla_scan v2 at a checkpoint boundary; verify gradient parity against the chunked reference.

Variables

  • ◆Kernel path (chunked MLX vs fused Metal v2)

Results

  • →Peak memory 23.88GB → 10.34GB (−57%).
  • →Throughput ~1,074 → ~1,500 tok/s (+40%).
  • →Gradient parity ~1e-7 relative error; loss continued cleanly, no NaN/Inf.

Observations

  • ·Bench (B=3, L=512, H=12, Dh=64): SSD fwd 2.30ms, fwd+bwd 17.34ms at 145MB; GLA fwd 1.41ms, fwd+bwd 12.06ms at 81MB.
  • ·v0.1 vs Python loop: up to 31.8× fwd+bwd speedup (GLA).

Conclusion

Kernels validated in production. All correctness and gradient gates passed.

Next Experiment

Use as the substrate for the HibbieFormer-HX helical core.