Research2026-09-21
Researchers introduced Critical-State RL to diagnose trainable states in multi-turn tool use. It separates action-dependent reward variation from downstream noise. The method uses nested sampling and contextual-bandit training. Experiments ran on the Berkeley Function Calling Leaderboard v4. Training selected responses improved performance by about 14 percentage points on missing-function tasks. Alternative state training left performance flat or worse. The approach also applies to repeat-call avoidance and memory management.
What it unlocks
Identifying which model calls in multi-turn tool use benefit from training.
- 14 percentage points performance gain on missing-function task
- 31 pages, 8 figures, 7 tables
Sources