Noesion Radarnoesion.ai
Research2026-08-03

Epoch AI published MirrorCode, a test that asks AI models to rebuild entire existing programs from scratch without seeing the original code, with results checked against hidden tests. It was built with METR and covers 25 programs across areas such as Unix tools, bioinformatics, interpreters and compression. The authors caution that models likely saw these open-source programs during training, which may inflate scores.

What changed

Most software engineering benchmarks measured short tasks such as fixing a bug or adding one feature, with spending capped at roughly $1-10 per attempt.

What it unlocks

Comparing models on whole-program rebuilds that run for days, using a published scaffold and 22 released target programs.

  • 25 target programs, 22 released open-source
  • one task cost $2,600 for a single run with AI working 19 days unattended
  • Claude Opus 4.7 rebuilt a ~16,000-line Go toolkit in 14 hours for $251
  • human estimate for that task: 2-17 weeks
  • leaderboard budget: 10 billion tokens and 7 days per attempt, 30 tasks run three times

What you need to act on it

  • substantial compute budget to run the benchmark

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.