Language models can abruptly switch between shallow pattern-following and transferable computation during pre-training. The authors call this behavior mode-hopping. It appears in hard accuracy and probability margins, not just evaluation scores. Mode-hopping persists even when plotting against pre-training FLOPs or tokens. It cannot be fixed by checkpoint averaging or single optimization steps. The authors hypothesize it stems from competition between shallow and generalizable circuits in capacity-bounded models. They built a cheap behavioral eval suite to detect it. The suite helped select intermediate checkpoints that outperformed later ones on reasoning and alignment tasks. It also showed that pre-training data selection can stabilize generalization dynamics. The suite is available on GitHub.
What changed
Language models were assumed to mature smoothly from pattern-matching to generalizable reasoning during pre-training.
What it unlocks
Selecting intermediate pre-training checkpoints that generalize better than final ones for reasoning and alignment tasks.
- 81% to 0% to 81.7% arithmetic accuracy across 0.04T tokens
- 36.3% vs 29.8% GPQA after math fine-tuning
- 53% vs 21% robustness to prefilling attacks
What you need to act on it
- access to intermediate pre-training checkpoints
- a toy eval suite to detect mode-hopping
Sources