TL;DR
Independent measurements are in for NVIDIA’s Nemotron 3.5 line. Artificial Analysis scores Super VL at 26.5 — the top US open-weight model — and Bolt 74B-A9B at 21.4, fourth among sub-100B models. NVIDIA’s own model card shows Bolt beating Qwen3.6 and Gemma 4 on agentic benchmarks like Terminal-Bench 2.1 and BrowseComp — but the card also discloses its fine-tuning targeted those exact benchmarks, and NVIDIA’s own speed table has Bolt trailing both rivals on tokens per second. The 2-bit compression holds up; the speed framing doesn’t.
The independent scores
Artificial Analysis put both released Nemotron 3.5 models through its Intelligence Index. Super VL landed at 26.5 — the highest score among US open-weight models, edging out Thinking Machines’ Inkling Small (25.7) and sitting ahead of NVIDIA’s own Nemotron 3 Ultra (23) and Meta’s Muse Glimmer (17). It delivers twice the intelligence of its predecessor, Nemotron 3 Super (13), on roughly 60% fewer output tokens per task (46k vs 114k), and its NVFP4 build is near-lossless versus BF16 — within 0.1 points, with no component moving more than 2.6.
Bolt 74B-A9B — the 74-billion-parameter model squeezed into 24GB — scored 21.4, placing fourth among models under 100B parameters and first among US models in that class. Against Nemotron 3 Ultra it leads on AutomationBench-AA and AA-Briefcase but trails on AA-LCR (70% vs 79%) and Humanity’s Last Exam (25% vs 28%). Artificial Analysis measured Super VL at roughly 630 output tokens per second on 100,000-token prompts — on a pre-release endpoint, so directional, not a guarantee. There is no independent speed figure for Bolt yet; it has no public API endpoint.
NVIDIA’s own benchmark table — and the caveat
The HuggingFace model card pits Bolt against Qwen3.6-35B-A3B and Gemma 4 26B-A4B: GPQA Diamond 86.43 vs 83.40 / 79.61, Terminal-Bench 2.1 50.98 vs 44.38 / 37.22, HLE (no tools) 25.30 vs 19.56 / 17.42, BrowseComp 66.75 vs 48.74 / 26.30. Bolt does not win everything: Qwen3.6 edges it on PinchBench (88.07 vs 87.82) and both rivals beat it on AA-Omniscience accuracy (19.47 / 22.17 vs 16.90).
The caveat, stated plainly because the card states it: Bolt’s fine-tuning used Kimi-K3-generated synthetic datasets targeting AIME, GPQA Diamond, Terminal-Bench 2.1, SciCode, AA-LCR, APEX, and LMArena. The benchmarks it wins are substantially the benchmarks it trained on. That does not invalidate the scores — targeted training is legitimate post-training — but it means they measure the training recipe as much as the architecture.
The speed claim vs NVIDIA’s own table
The model card pitches Bolt as ‘faster than most popular dense models while smarter than most popular MoE models with a similar memory footprint.’ The dense half holds: on NVIDIA’s technical-blog table (AIPerf SPEED-Bench, RTX Spark), Bolt hits 83.99 tokens/sec against 28.26 for dense Qwen3.8 27B. But against the MoE peers it compares itself to on accuracy, it is slower: Qwen3.6-35B-A3B at 96.63 and Gemma 4 26B-A4B at 100.56 tokens/sec on the same table.
To be fair, NVIDIA’s stated design target was at least 80 tokens/sec, and 84 clears it — the engineering goal was met. What doesn’t survive is the ‘much faster’ framing when the company’s own numbers put it third of three on throughput.
Local speed, measured independently
The community benchmark site llm-speed ran NVIDIA’s official 2-bit GGUF (22.0 GB) under llama.cpp on consumer cards, one run each at 4K context: 87.8 tokens/sec on an RTX 3090, 107.3 on a 4090, 154.7 on a 5090. On the same 5090 and build, dense Qwen3.8-27B Q4_K_M managed 75.2 — Bolt decodes about twice as fast because it reads only ~9B active parameters per token from a 2-bit file.
The 24GB claim checks out for single-user use: peak VRAM was about 22.0 of 24.6 GB on the 4090. Two practical notes: Ollama 0.40.4 cannot load the model yet (its bundled llama.cpp lacks support — use llama.cpp build 620d2225 or later), and there are still no measured Mac, RTX Spark, or Ollama numbers.
Discrepancies worth knowing
AA-LCR: NVIDIA reports 64.06; Artificial Analysis measured 70%. MCP-Atlas: NVIDIA’s card reports 52.20 for Bolt and discloses in a footnote that Qwen’s own reported 62.8 became 51.2 under NVIDIA’s measurement — the card is honest about the gap, which is to its credit. Context length is listed as 256K on the model card and by Artificial Analysis, but Unsloth’s run-locally docs page says 128K (their own HuggingFace card says 256K — an internal inconsistency on their side).
And a naming note, because NVIDIA’s lineup is getting confusing: this is Bolt 74B-A9B, the 24GB local-agent model. It is not Bolt E4B (the 4GB edge model, due October 23) and not Lightning 30B (the August high-throughput release).