Ian Barber examined a research paper on test-time training and argued the label covers two different things. The paper is titled "Test-Time Training with KV Binding Is Secretly Linear Attention". Its authors say such designs can be rewritten as a learned form of linear attention. That contradicts the usual view that these layers memorise key-value pairs while running. Barber distinguished slow weights, which are updated by ordinary training, from fast weights, produced during a single pass. The paper trains fast weights, using a matrix that does not grow with sequence length. Barber noted the fast weights reset for each new input sequence. So the approach does not deliver continual learning, in his reading.
Sources