NVIDIA squeezed a 74B agentic model into 24GB — and open-sourced the whole thing

NVIDIA announced on October 13 that Nemotron 3.5 Bolt 74B-A9B 24GB releases Thursday, October 15 — one day before RTX Spark laptops launch on October 16. It is a mixture-of-experts design: 74 billion parameters total, only 9 billion active per query, packed into 24GB using 2-bit quantization with quantization-aware training to preserve quality. NVIDIA positions it as faster than most popular dense models while smarter than comparable MoE models at a similar memory footprint.

At launch it runs through llama.cpp, vLLM, Ollama, Unsloth, and LM Studio, and it is wired directly into agent frameworks like Hermes and OpenClaw. Like other Nemotron models, it ships fully open source — weights, training data, and recipes — so developers can fine-tune and customize their own agents on top of it.

The point is not another leaderboard score; it is where the model runs. A 74B-class agent model on a laptop means always-on personal agents without routing everything through a cloud API bill. NVIDIA says local LLM downloads are already five times higher than all of last year, and Bolt 74B is purpose-built for that shift.

One honest caveat: the quantization and speed-versus-intelligence claims are NVIDIA's own. Independent benchmarks will tell the real story once the weights land on October 15.

NVIDIA PAIR (Personal AI Router) officially moved out of beta on October 13. It is a free, open-source tool that discovers compatible PCs on your local network and routes whole AI inference requests to whichever machine has spare capacity. It does not pool GPU memory across machines — each request runs on one node — but parallel agents on separate machines is exactly what agentic workflows need.

The official release adds llama.cpp support, direct model downloads from Hugging Face, customizable inference engine settings, and RTX Spark support, on Windows, Linux, and macOS. In the month since the beta release, the community has already contributed more than 10 features and fixes to the project.

If you are getting an RTX Spark laptop on Friday, Bolt 74B lands the day before: it runs through Ollama, LM Studio, llama.cpp, vLLM, and Unsloth at launch, NVIDIA says. If you already have a couple of RTX PCs sitting around, grab PAIR and point your agent tooling at it — the Ollama-compatible endpoints mean no code changes.

Also new this week: llama.cpp released a Windows-native companion app ('Llama Windows') so you can download and run models from the system tray without the command line, and NVIDIA's open-source Switchyard library routes queries between local models and cloud frontier models. Keep the frontier calls for the hard stuff; run the routine agent steps local and free.