Research2026-10-16
Researchers introduced PresenceBench to test model adaptation to user state. It measures two properties: receptivity and specificity. Receptivity tests if models use state to improve responses. Specificity tests if improvements depend on the correct state. Four models were evaluated across 30 scenarios. All models showed point estimates above chance for receptivity. Specificity results were inconclusive. Policy delivery reduced cost and token use. Safety failures were rare and not significantly higher under policy delivery. The study reports the instrument and frozen results for future confirmatory work.
What it unlocks
Measuring whether large language models adapt responses to a user's functional state.
- 4 models tested
- 18 single-turn scenarios
- 12 multi-turn scenarios
- 13 safety-floor failures in 360 cases
What you need to act on it
- a synthetic state signal or response policy
- human paired comparison scoring
Sources