CodeRabbit published benchmark results for Anthropic's newly released Claude Opus 5 on automated code review, using roughly 100 error patterns from real open-source pull requests. At its highest effort setting the model produced a more precise set of actionable comments than CodeRabbit's current production mix, but caught fewer of the known bugs and generated about four times as many low-value nitpicks. CodeRabbit recommends it as a second reviewer rather than the only one, and rates it more highly for writing code than for reviewing it.
What changed
Opus 4.8 sat close to CodeRabbit's production baseline as the balanced option in the family, and showed quality degradation past 200k tokens.
What it unlocks
Using Opus 5 as a second, precision-oriented reviewer alongside a model that catches more issues, with a clearer view of the trade-off before rolling it out.
- actionable precision 39.3% vs 35.2% baseline
- known issues caught 55.2% vs 61.1% baseline
- 92 nitpicks vs 23 for baseline
- full-stream precision 28.6% vs 32.8%
- ~60.5k input and ~9.5k output tokens per review call vs ~40.5k and ~5.8k
- 1M token context window, default and maximum
What you need to act on it
- access to Claude Opus 5
- a review setup that can route between models and filter comment volume
Sources