NVIDIA released Nemotron 3.5 Bolt 74B-A9B 24GB. It is a Mixture-of-Experts hybrid Mamba-Transformer model. It fits in 24GB of memory. It achieves at least 80 tokens per second on RTX Spark. It was compressed from Nemotron 3.5 Super VL using Star Elastic. Experts were quantized to 2.0625 bits per element. A LiLiCorr speculative drafter boosts speed to 84 TPS. It outperforms Qwen3.6 35B and Gemma 4 26B on agentic benchmarks. It ranks first on PinchBench and second on WebArena and BrowseComp. Checkpoints and drafters are open-source on Hugging Face.
What changed
Larger models required more memory or lower quantization precision for local deployment.
What it unlocks
Running a high-performance agent model locally on RTX or DGX Spark with 24GB memory.
- 24GB memory footprint
- 84 TPS on RTX Spark
- 2.0625 bits per element quantization
- #1 in PinchBench, #2 in WebArena and BrowseComp
What you need to act on it
- RTX Spark or DGX Spark hardware
- llama.cpp or vLLM for inference
Sources