Research2026-10-15

Reinforcement learning fine-tuning changes only 5–30% of a model’s parameters. Small updates and sparse updates are different claims. On the model’s own samples the score averages to zero. The expected policy gradient is the covariance between the learning signal and the score. The Fisher information is the score’s variance. A parameter can only move where its Fisher is large. Whether it does move depends on the reward. Off-policy, the same holds once the score is centered under the sampling distribution. The Fisher at one position is the variance of logit sensitivities. Across positions the Fisher adds up. None of this requires on-policy sampling.

What it unlocks

Understanding why post-training updates in language models are sparse and small.

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.