Are Large Reasoning Models Interruptible?
Large Reasoning Models (LRMs) excel at complex reasoning but are
traditionally evaluated in static, "frozen world" settings: model responses are
assumed to be instantaneous, and the context of a request is presumed to be
immutable over the duration of the response. While generally true for
short-term tasks, the "frozen world" assumption breaks down in modern reasoning
tasks such as assistive programming, where models may take hours to think
through problems and code may change dramatically from the time the model
starts thinking to the model's final output. In this work, we challenge the
frozen world assumption and evaluate LRM robustness under two realistic dynamic
scenarios: interruptions, which test the quality of the model's partial outputs
on a limited budget, and dynamic context, which tests model adaptation to
in-flight changes. Across mathematics and programming benchmarks that require
long-form reasoning, static evaluations consistently overestimate robustness:
even state-of-the-art LRMs, which achieve high accuracy in static settings, can
fail unpredictably when interrupted or exposed to changing context, with
performance dropping by up to 60% when updates are introduced late in the
reasoning process. Our analysis further reveals several novel failure modes,
including reasoning leakage, where models fold the reasoning into their final
answer when interrupted; panic, where under time pressure models abandon
reasoning entirely and return incorrect answers; and self-doubt, where
performance degrades while incorporating updated information.
68edc9b7de1fee572713a919
Tsung-Han Wu
claimed_verified
2025-10-14T07:29:59.528Z
644a767044b75fd95805d232
https://cdn-avatars.huggingface.co/v1/production/uploads/644a767044b75fd95805d232/vHA2vI_B3CpXapdBEwspB.jpeg
Patrick (Tsung-Han) Wu
user
tsunghanwu
68edc9b7de1fee572713a91a
Mihran Miroyan
claimed_verified
2025-10-21T14:06:27.891Z
647a99bd61e1252d761ae6ed
https://cdn-avatars.huggingface.co/v1/production/uploads/647a99bd61e1252d761ae6ed/CI67hb26bP9l1yIA_NIGE.jpeg
Mir Miroyan
user
mmiroyan
68edc9b7de1fee572713a91b
David M. Chan
68edc9b7de1fee572713a91c
Trevor Darrell
68edc9b7de1fee572713a91d
Narges Norouzi
68edc9b7de1fee572713a91e
Joseph E. Gonzalez
Are Large Reasoning Models Interruptible?
Large Reasoning Models (LRMs) excel at complex reasoning but are
traditionally evaluated in static, "frozen world" settings: model responses are
assumed to be instantaneous, and the context of a request is presumed to be
immutable over the duration of the response. While generally true for
short-term tasks, the "frozen world" assumption breaks down in modern reasoning
tasks such as assistive programming, where models may take hours to think
through problems and code may change dramatically from the time the model
starts thinking to the model's final output. In this work, we challenge the
frozen world assumption and evaluate LRM robustness under two realistic dynamic
scenarios: interruptions, which test the quality of the model's partial outputs
on a limited budget, and dynamic context, which tests model adaptation to
in-flight changes. Across mathematics and programming benchmarks that require
long-form reasoning, static evaluations consistently overestimate robustness:
even state-of-the-art LRMs, which achieve high accuracy in static settings, can
fail unpredictably when interrupted or exposed to changing context, with
performance dropping by up to 60% when updates are introduced late in the
reasoning process. Our analysis further reveals several novel failure modes,
including reasoning leakage, where models fold the reasoning into their final
answer when interrupted; panic, where under time pressure models abandon
reasoning entirely and return incorrect answers; and self-doubt, where
performance degrades while incorporating updated information.
Large Reasoning Models evaluated in dynamic scenarios with interruptions and changing context show significant performance drops compared to static evaluations.
Large Reasoning Models frozen world assumption interruptions dynamic context long-form reasoning reasoning leakage panic self-doubt