Research2026-08-28
Researchers published the code behind a study in which a Claude-based agent proposes and tests training methods to reduce ten types of AI misbehaviour, including flattery, jailbreaks, deception and privacy violations. The release covers the evaluation suite, the full automated loop, and a template for applying the same approach to other measurable tasks. Real evaluation runs need a Linux machine with an NVIDIA GPU, and no training backend is bundled.
What changed
The harness and benchmark suite behind the study were not publicly available.
What it unlocks
Rerunning the ten safety evaluations on a chosen model, or pointing the same automated loop at any task that has one scored and one hidden benchmark.
- 10 alignment failures covered
- 3-5 scored benchmarks per failure
- 1 hidden benchmark per failure
What you need to act on it
- Linux machine with an NVIDIA GPU and CUDA 12
- Python 3.12 or newer
- Hugging Face token plus OpenAI or Anthropic keys for the scoring models
- own training code for the full research loop
Sources