Claude Code added build-eval and hillclimb commands to the claude-api skill. build-eval guides users through creating evaluations with production-like tasks. It prioritizes real transcripts, bug reports, and hand-written cases. It proposes the simplest grader that fits the output type. It validates the grader by scoring sample cases for user review. It runs a baseline and reports scores with confidence intervals. hillclimb iteratively improves an application against a chosen evaluation. It splits cases into train and test sets to detect overfitting. It proposes one change per round and reverts if test scores do not improve. It stops when progress stalls and suggests next steps. The skill warns if baseline scores are near saturation. It checks for grader consistency, infrastructure noise, and headroom. It avoids pasting failures into prompts to prevent overfitting. It keeps answers structurally out of the model’s reach.
What it unlocks
Automatically design evaluations and iteratively improve applications against them using guided commands in Claude Code.
What you need to act on it
- Claude Code with the claude-api skill installed
Sources