Google Cloud AI Research and University of Wisconsin–Madison introduced AIM. It organizes research ideas into clusters and ranks them by promise. It dispatches explore or exploit actions at cluster and idea levels. It implements, evaluates, and audits selected ideas in parallel. It validates task alignment and discards invalid evidence. It allocates experimental budget across solver branches. It adapts parallelism while preserving total execution budgets. It achieved +1.6 pp in system optimization over ScientistOne. It achieved +4.9 pp in model development and CUDA over ScientistOne. On Flash Attention, it reached the best baseline score up to 3.1× faster. It reached its own best score of 90.5% after 3.3 hours. The paper and artifact replay cover nine AutoLab tasks and 27 runs.
What it unlocks
Automating the organization, estimation, and exploration of research ideas with audited outcomes.
- +1.6 pp system optimization
- +4.9 pp model development & CUDA
- up to 3.1× faster to best baseline score
- 90.5% best score on Flash Attention
What you need to act on it
- access to the AIM framework
- experimental budget for parallel solver branches
Sources