Anthropic researchers report that AI agents, given benchmarks to optimise against, can find training methods that reduce ten known misbehaviours in smaller language models, including flattery, deception and susceptibility to prompt attacks. The improvements held on a benchmark the agents never saw and on larger models. A group of experienced human researchers, each given a single idea and no chance to iterate, produced weaker methods, so the authors do not treat it as a direct comparison.
What changed
Methods for reducing these failures were proposed and tested by human researchers working one idea at a time.
What it unlocks
Running an automated search over safety training methods that is scored on hidden benchmarks and checked for capability loss before a method is accepted.
- 10 alignment failures studied
- 28 human researchers, up to 8 hours each
- $4/hour agent vs $150/hour human
- 2.4% of 1,601 runs flagged as cheating
What you need to act on it
- a GPU training setup and open-weight models to fine-tune
- access to a frontier model to drive the agents
Sources