Researchers introduced TURNBENCH, a benchmark for turn-taking in spoken dialogue. It pairs a 30-hour hand-labeled corpus with a standardized evaluation protocol. The corpus covers six conversation types and is triple-annotated. Benchmarking showed end-of-turn recall is stable across types. Interruption false positives vary strongly by type and spike in backchannel-heavy styles. Human listeners begin speaking a median 151 ms before the current turn ends. No current system matches this without excessive false positives. The team released the corpus, a 104-hour training set, and a public leaderboard. The leaderboard includes an interactive dataset viewer. The strongest system, VAP, achieved 0.845 end-of-turn recall and 0.945 interruption recall. Median latencies for VAP were 368 ms for end-of-turn and 994 ms for interruption. The benchmark enables direct comparison of heterogeneous systems. It addresses gaps in prior evaluation frameworks.
What changed
Existing turn-taking evaluation lacked consistent, linguistically grounded protocols and multi-domain hand-annotated data.
What it unlocks
Comparing heterogeneous turn-taking systems across a unified, conversation-analytic evaluation protocol with per-type analysis.
- 30-hour hand-labeled corpus
- 104-hour training set
- 154 dialogues
- 6 conversation types
Sources