Researchers tested whether AI agents can perform open-ended scientific discovery. They used Station, an open-world environment simulating a scientific ecosystem. Agents were given research questions from three ICLR papers but no results or web access. Station rediscovered 62.7% of the original findings on average. Codex Multiagent-v2 rediscovered 15.4% and AI Scientist-v2 14.4-20.6%. Adding Supervisor and Meta Reflection mechanisms improved research coverage and continuity. Agents also made discoveries matching post-cutoff research in two open-ended tasks. The paper concludes a suitable environment enables meaningful progress in open-ended discovery.
What changed
AI systems required well-defined metrics to make scientific progress.
What it unlocks
AI agents can autonomously rediscover scientific findings in open-ended tasks without predefined metrics.
- 62.7% of criteria rediscovered
- 15.4% for Codex Multiagent-v2
- 14.4-20.6% for AI Scientist-v2
What you need to act on it
- an open-world environment like Station
- Supervisor mechanism and periodic Meta Reflection
Sources