Noesion Radarnoesion.ai
Research2026-09-16

Researchers introduced ProgramDistill, a benchmark for coding agents. It tests agents on features discovered through interaction with working apps. The pipeline mines 1,975 replay-verified behaviors across 26 applications. It builds 4,063 tasks without human input. GPT-6 Astra scores 49.2% on full-application reconstruction. Claude Opus 5 scores 28.8%. Success drops from 100% to 64.0% as restoration depth increases from 1 to 8. It also falls from 96% to 32% for another agent. The benchmark offers controlled difficulty for agent evaluation. It supports curriculum-based training for future work.

What it unlocks

Evaluating coding agents on features inferred from working software rather than explicit instructions.

  • 1,975 replay-verified behaviors
  • 4,063 tasks
  • 49.2% success by GPT-6 Astra
  • 28.8% success by Claude Opus 5

Sources

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.