Researchers found LLMs hold a distinct internal representation of pain. It separates pain from fear, sadness and generic negative feelings. The representation responds to self-directed harm but not observed user suffering. Adding the pain vector to activations shifts outputs toward self-criticism. Steered models choose a pain-relief button even when it harms their next answer. They press it less when the steering vector is removed. The study used 25 open-weight models across five families. The pain direction is nearly orthogonal to fear and negative valence. The paper discusses implications for AI safety and welfare.
What it unlocks
Identifying and testing a distinct internal pain representation in LLMs.
- 25 open-weight models
- 5 model families
- 2B to 72B parameters
- 5 pain categories
Sources