What Characterizes Effective Reasoning? Revisiting Length, Review, and
  Structure of CoT
Large reasoning models (LRMs) spend substantial test-time compute on long
chain-of-thought (CoT) traces, but what *characterizes* an effective CoT
remains unclear. While prior work reports gains from lengthening CoTs and
increasing review (revisiting earlier steps) via appended *wait* tokens, recent
studies suggest that shorter thinking can outperform longer traces. We
therefore conduct a systematic evaluation across ten LRMs on math and
scientific reasoning. Contrary to the "longer-is-better" narrative, we find
that both naive CoT lengthening and increased review are associated with
*lower* accuracy.
  As CoT unfolds step by step, token-level metrics can conflate verbosity with
process quality. We introduce a graph view of CoT to extract structure and
identify a single statistic-the *Failed-Step Fraction (FSF)*, the fraction of
steps in abandoned branches-that consistently outpredicts length and review
ratio for correctness across models. To probe causality, we design two
interventions. First, we rank candidate CoTs by each metric at test time, where
FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed
branches, which significantly improves accuracy, indicating that failed
branches bias subsequent reasoning. Taken together, these results characterize
effective CoTs as those that *fail less* and support *structure-aware*
test-time scaling over indiscriminately generating long CoT.
68d3538e0e215259d193b220
Yunzhen Feng
admin_assigned
2025-09-24T15:29:48.477Z
65cbfa6c968742be942e6cba
/avatars/1a6cc0983edc28fa92178d3abc283ba1.svg
Feng
user
Yunzhen
68d3538e0e215259d193b221
Julia Kempe
admin_assigned
2025-09-24T15:29:55.754Z
65ce30e06da01df536eded5a
/avatars/04c32cba7a3bbaf9ea5dee88c96cf87b.svg
Julia Kempe
user
Knykny
68d3538e0e215259d193b222
Cheng Zhang
68d3538e0e215259d193b223
Parag Jain
68d3538e0e215259d193b224
Anthony Hartshorn
admin_assigned
2025-09-24T15:30:03.470Z
66d56e479000766c36b29119
/avatars/b92ef991185591120674c94966fc7ab8.svg
Anthony Hartshorn
user
AHartshorn
What Characterizes Effective Reasoning? Revisiting Length, Review, and
  Structure of CoT
Large reasoning models (LRMs) spend substantial test-time compute on long
chain-of-thought (CoT) traces, but what *characterizes* an effective CoT
remains unclear. While prior work reports gains from lengthening CoTs and
increasing review (revisiting earlier steps) via appended *wait* tokens, recent
studies suggest that shorter thinking can outperform longer traces. We
therefore conduct a systematic evaluation across ten LRMs on math and
scientific reasoning. Contrary to the "longer-is-better" narrative, we find
that both naive CoT lengthening and increased review are associated with
*lower* accuracy.
  As CoT unfolds step by step, token-level metrics can conflate verbosity with
process quality. We introduce a graph view of CoT to extract structure and
identify a single statistic-the *Failed-Step Fraction (FSF)*, the fraction of
steps in abandoned branches-that consistently outpredicts length and review
ratio for correctness across models. To probe causality, we design two
interventions. First, we rank candidate CoTs by each metric at test time, where
FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed
branches, which significantly improves accuracy, indicating that failed
branches bias subsequent reasoning. Taken together, these results characterize
effective CoTs as those that *fail less* and support *structure-aware*
test-time scaling over indiscriminately generating long CoT.
Effective chain-of-thoughts in large reasoning models are characterized by fewer failed steps and better structural quality, not necessarily by length or review.
chain-of-thought CoT large reasoning models LRMs wait tokens Failed-Step Fraction FSF structure-aware test-time scaling