Research2026-08-21
Security vendor Aikido benchmarked ten AI models on rediscovering 32 recently disclosed software vulnerabilities, running each model three times. DeepSeek V4 Pro found the most once results from all three runs were combined, ahead of the leading closed models, and repeating a cheaper model cost far less than a single expensive pass. The open models also produced the most false leads, so the savings shift work to whatever filters the results.
What changed
In the same team's earlier benchmark, open-weight models trailed the closed frontier on vulnerability discovery.
What it unlocks
Running a cheaper model three times over the same code to get broader vulnerability coverage than one pass of a top-priced model.
- 28 of 32 vulns pooled, DeepSeek V4 Pro
- 3 DeepSeek Pro runs ~$295
- single frontier pass ~$450-$590
- 3 Flash runs $108, 24 found
What you need to act on it
- an agent setup that pools and filters findings across repeated runs
- API access to the models tested
Sources