Researchers proposed Matryoshka Attribution to frame attribution as identifying nested subsets of internal components. The method uses a differentiable sigmoid top-k operator to learn masks. Training randomises k over all sparsities to produce a learned ordering by attribution score. MAttr tops the Mechanistic Interpretability Benchmark leaderboard. It identifies sparse and task-transferrable circuits across varying circuit bases. The method can be trained with reinforcement learning to pinpoint weight changes. Restoring 1% of Llama 3.1 8B Instruct weights to base state removed refusals while keeping capabilities. The paper presents interpretability as a learnable objective solvable with gradient descent.

• 1% of Llama 3.1 8B Instruct weights restored
• number 1 on Mechanistic Interpretability Benchmark