SemiAnalysis published verified agentic inference results for NVIDIA Rubin. Rubin achieved 67x better throughput per total cost of ownership than GB300. It also delivered 7x better token throughput per megawatt than Blackwell. Rubin earned over 2x more profit per gigawatt than Blackwell. The benchmark used was AgentX, an industry standard for agentic workloads. Results were measured on early pre-release software. Performance is expected to improve as software matures. Rubin outperformed GB300 in P90 interactivity and latency at higher throughput. The upgrade remains justified even at higher rental costs. At 80 TPS, Rubin produced 62% more tokens for the same rental TCO. Across high interactivity scenarios, Rubin achieved up to 16x more tokens per rental TCO.
What changed
Blackwell was the prior performance per watt and TCO baseline for large-scale inference.
What it unlocks
Generating far more tokens per dollar spent on compute for agentic inference workloads.
- 67x throughput per TCO vs GB300
- 2x profit per gigawatt vs Blackwell
- 7x token throughput per megawatt vs Blackwell
- $8.5/hr/chip Rubin rental cost
What you need to act on it
- early pre-release TRTLLM software
- production SKU of 2300W TDP & 1.5TB CPU LPDDR5X per tray
Sources