Fractured Chain-of-Thought Reasoning
Inference-time scaling techniques have significantly bolstered the reasoning
capabilities of large language models (LLMs) by harnessing additional
computational effort at inference without retraining. Similarly,
Chain-of-Thought (CoT) prompting and its extension, Long CoT, improve accuracy
by generating rich intermediate reasoning trajectories, but these approaches
incur substantial token costs that impede their deployment in latency-sensitive
settings. In this work, we first show that truncated CoT, which stops reasoning
before completion and directly generates the final answer, often matches full
CoT sampling while using dramatically fewer tokens. Building on this insight,
we introduce Fractured Sampling, a unified inference-time strategy that
interpolates between full CoT and solution-only sampling along three orthogonal
axes: (1) the number of reasoning trajectories, (2) the number of final
solutions per trajectory, and (3) the depth at which reasoning traces are
truncated. Through extensive experiments on five diverse reasoning benchmarks
and several model scales, we demonstrate that Fractured Sampling consistently
achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling
gains in Pass@k versus token budget. Our analysis reveals how to allocate
computation across these dimensions to maximize performance, paving the way for
more efficient and scalable LLM reasoning.
682c12290f622b7afc1fc98f
Baohao Liao
admin_assigned
2025-05-20T08:33:43.331Z
62c414354ce7250560a1f67f
/avatars/28fd73973d1703c84f4f59644fef8a80.svg
Baohao Liao
user
baohao
682c12290f622b7afc1fc990
Hanze Dong
admin_assigned
2025-05-20T08:33:50.212Z
63a3ff69f91ad3ea5703841d
/avatars/69227c4bce01d33747c1377b6f9672db.svg
Hanze Dong
user
hendrydong
682c12290f622b7afc1fc991
Yuhui Xu
admin_assigned
2025-05-20T08:33:37.033Z
6602869253a0518b2a98cafd
/avatars/c14b5953a716f42c83ad28147f8308ae.svg
Yuhui Xu
user
yuhuixu
682c12290f622b7afc1fc992
Doyen Sahoo
admin_assigned
2025-05-20T08:33:57.766Z
65f84fd980481173afd91233
/avatars/6ac7bd6beba24d1476c5179b88c9e3fa.svg
Doyen
user
doyensahoo
682c12290f622b7afc1fc993
Christof Monz
682c12290f622b7afc1fc994
Junnan Li
admin_assigned
2025-05-20T08:34:18.715Z
61f9d3b54ac99e8a1bae85f4
/avatars/ac47d13204dd22452e4bc46e280842d5.svg
JunnanLi
user
JunnanLi
682c12290f622b7afc1fc995
Caiming Xiong
admin_assigned
2025-05-20T08:34:27.628Z
649dbcc4e0fff1ed099dc80a
/avatars/c87c273ca628dbcddccbf1ee19b2ce33.svg
Caiming Xiong
user
cxiong
Fractured Chain-of-Thought Reasoning
Inference-time scaling techniques have significantly bolstered the reasoning
capabilities of large language models (LLMs) by harnessing additional
computational effort at inference without retraining. Similarly,
Chain-of-Thought (CoT) prompting and its extension, Long CoT, improve accuracy
by generating rich intermediate reasoning trajectories, but these approaches
incur substantial token costs that impede their deployment in latency-sensitive
settings. In this work, we first show that truncated CoT, which stops reasoning
before completion and directly generates the final answer, often matches full
CoT sampling while using dramatically fewer tokens. Building on this insight,
we introduce Fractured Sampling, a unified inference-time strategy that
interpolates between full CoT and solution-only sampling along three orthogonal
axes: (1) the number of reasoning trajectories, (2) the number of final
solutions per trajectory, and (3) the depth at which reasoning traces are
truncated. Through extensive experiments on five diverse reasoning benchmarks
and several model scales, we demonstrate that Fractured Sampling consistently
achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling
gains in Pass@k versus token budget. Our analysis reveals how to allocate
computation across these dimensions to maximize performance, paving the way for
more efficient and scalable LLM reasoning.
Fractured Sampling optimizes inference in large language models by balancing token usage and accuracy through truncated reasoning trajectories.
inference-time scaling large language models Chain-of-Thought (CoT) Long CoT truncated CoT Fractured Sampling reasoning trajectories final solutions reasoning traces Pass@k token budget