NVIDIA released a compressed version of its Nemotron 3.5 Lightning model, a 30-billion-parameter open-weights system that only activates 3 billion parameters at a time and handles up to one million tokens of text at once. It is licensed for commercial use and can be served on a single H100 or a DGX Spark desktop machine. Deployment requires NVIDIA hardware and one of three supported serving frameworks on Linux.
What changed
Earlier Nemotron Lightning releases did not ship a quantized checkpoint tuned for single-GPU and desktop-class NVIDIA hardware with these speculative decoding drafters.
What it unlocks
Running a long-context reasoning and tool-calling model with a million-token window on a single H100 or one DGX Spark desktop unit, under a commercial-use open licence.
- 30B total, 3B active parameters
- 1M token context
- 51.6 BF16 vs 52.8 NVFP4 SWE-bench Verified
- runs on 1x H100 or DGX Spark
What you need to act on it
- NVIDIA Blackwell, Hopper or Ampere GPU
- Linux with vLLM, TensorRT-LLM or SGLang
- self-hosting capability
Sources