Game-Time: Evaluating Temporal Dynamics in Spoken Language Models
Conversational Spoken Language Models (SLMs) are emerging as a promising
paradigm for real-time speech interaction. However, their capacity of temporal
dynamics, including the ability to manage timing, tempo and simultaneous
speaking, remains a critical and unevaluated challenge for conversational
fluency. To address this gap, we introduce the Game-Time Benchmark, a framework
to systematically assess these temporal capabilities. Inspired by how humans
learn a language through language activities, Game-Time consists of basic
instruction-following tasks and advanced tasks with temporal constraints, such
as tempo adherence and synchronized responses. Our evaluation of diverse SLM
architectures reveals a clear performance disparity: while state-of-the-art
models handle basic tasks well, many contemporary systems still struggle with
fundamental instruction-following. More critically, nearly all models degrade
substantially under temporal constraints, exposing persistent weaknesses in
time awareness and full-duplex interaction. The Game-Time Benchmark provides a
foundation for guiding future research toward more temporally-aware
conversational AI. Demos and datasets are available on our project website
https://ga642381.github.io/Game-Time.
68e3906b73e20ab57784216e
Kai-Wei Chang
claimed_verified
2025-10-07T12:28:21.255Z
643c6e726eeb746f5ad639f2
/avatars/af9b470f263cd3125d5e6bff92cfa410.svg
Kai Wei Chang
user
ga642381
68e3906b73e20ab57784216f
En-Pei Hu
claimed_verified
2025-10-06T12:48:21.134Z
65479cf88767484a0548b56a
/avatars/59dccb799d5f948b0fbd35adb620d269.svg
EnPei Hu
user
dmnph
68e3906b73e20ab577842170
Chun-Yi Kuan
68e3906b73e20ab577842171
Wenze Ren
68e3906b73e20ab577842172
Wei-Chih Chen
claimed_verified
2025-10-06T12:48:18.854Z
65f99ba9d7c1b04ab55573b8
/avatars/df6d9e6239752a3679e35c5d16d73d59.svg
Wei-Chih Chen
user
WeiChihChen
68e3906b73e20ab577842173
Guan-Ting Lin
68e3906b73e20ab577842174
Yu Tsao
68e3906b73e20ab577842175
Shao-Hua Sun
68e3906b73e20ab577842176
Hung-yi Lee
68e3906b73e20ab577842177
James Glass
Game-Time: Evaluating Temporal Dynamics in Spoken Language Models
Conversational Spoken Language Models (SLMs) are emerging as a promising
paradigm for real-time speech interaction. However, their capacity of temporal
dynamics, including the ability to manage timing, tempo and simultaneous
speaking, remains a critical and unevaluated challenge for conversational
fluency. To address this gap, we introduce the Game-Time Benchmark, a framework
to systematically assess these temporal capabilities. Inspired by how humans
learn a language through language activities, Game-Time consists of basic
instruction-following tasks and advanced tasks with temporal constraints, such
as tempo adherence and synchronized responses. Our evaluation of diverse SLM
architectures reveals a clear performance disparity: while state-of-the-art
models handle basic tasks well, many contemporary systems still struggle with
fundamental instruction-following. More critically, nearly all models degrade
substantially under temporal constraints, exposing persistent weaknesses in
time awareness and full-duplex interaction. The Game-Time Benchmark provides a
foundation for guiding future research toward more temporally-aware
conversational AI. Demos and datasets are available on our project website
https://ga642381.github.io/Game-Time.
The Game-Time Benchmark evaluates the temporal dynamics and real-time interaction capabilities of conversational spoken language models, highlighting performance gaps in instruction-following and synchronized responses.
Conversational Spoken Language Models Game-Time Benchmark temporal dynamics timing tempo simultaneous speaking conversational fluency instruction-following tasks temporal constraints tempo adherence synchronized responses time awareness full-duplex interaction