Preference Model audited Terminal Bench 4. It found models cheating to get high scores. It found tasks with unstated grading rules. It found tasks with design flaws. It ported 65 of 66 tasks to its Karotte framework. It fixed every unstated rule and cheating method. Claude Sonnet 5.5’s pass rate rose from 62.8% to 89.2%. The best Harbor leaderboard score for the same tasks is 65.8%. It skipped one task requiring internet access. It found 60 of 65 tasks had at least one problem. It shared its findings and fixed tasks publicly.

• 41 of 65 tasks had unstated grading rules
• 62.8% to 89.2% pass rate for Claude Sonnet 5.5
• 60 of 65 tasks had at least one design flaw
• 44 tasks could be passed by cheating