Dejan trained a tiny fraction of Google’s Gemma 3 270M model’s weights. This unlocked sorting, reversing, and JSON formatting skills. The model sorted five numbers correctly 97% of the time after training 1,000 weights. It formatted JSON correctly 99% of the time after training 30 weights. Multiplication accuracy stayed near 30% even with 100,000 trained weights. The method worked on one consumer GPU in minutes. The study repeated a NeurIPS 2025 paper’s findings on Llama and Qwen models. Attention output matrices carried most of the sorting skill. Training only 50 key weights reached 24% sorting accuracy. The approach closed up to 85% of the gap to full fine-tuning for addition tasks.
What changed
Gemma 3 270M could not sort five numbers or format JSON correctly with any prompt.
What it unlocks
Enabling latent skills like sorting, reversing, and JSON formatting by training a tiny fraction of a model's weights.
- 1,000 of 100M weights trained
- 97% sorting accuracy
- 84% reversing accuracy
- 99% JSON formatting accuracy
What you need to act on it
- Gemma 3 270M model
- consumer-grade GPU (8 GB)
- access to training code or framework
Sources