Beyond the Last Answer: Your Reasoning Trace Uncovers More than You
  Think
Large Language Models (LLMs) leverage step-by-step reasoning to solve complex
problems. Standard evaluation practice involves generating a complete reasoning
trace and assessing the correctness of the final answer presented at its
conclusion. In this paper, we challenge the reliance on the final answer by
posing the following two questions: Does the final answer reliably represent
the model's optimal conclusion? Can alternative reasoning paths yield different
results? To answer these questions, we analyze intermediate reasoning steps,
termed subthoughts, and propose a method based on our findings. Our approach
involves segmenting a reasoning trace into sequential subthoughts based on
linguistic cues. We start by prompting the model to generate continuations from
the end-point of each intermediate subthought. We extract a potential answer
from every completed continuation originating from different subthoughts. We
find that aggregating these answers by selecting the most frequent one (the
mode) often yields significantly higher accuracy compared to relying solely on
the answer derived from the original complete trace. Analyzing the consistency
among the answers derived from different subthoughts reveals characteristics
that correlate with the model's confidence and correctness, suggesting
potential for identifying less reliable answers. Our experiments across various
LLMs and challenging mathematical reasoning datasets (AIME2024 and AIME2025)
show consistent accuracy improvements, with gains reaching up to 13\% and 10\%
respectively. Implementation is available at:
https://github.com/hammoudhasan/SubthoughtReasoner.
6813376e2379095ee572f2f8
Hasan Abed Al Kader Hammoud
admin_assigned
2025-05-05T08:01:19.886Z
642b51385bf2355d02a23d15
/avatars/87985347643b2647555f2453fa4d94fb.svg
Hasan Abed Al Kader Hammoud
user
hammh0a
6813376e2379095ee572f2f9
Hani Itani
admin_assigned
2025-05-05T08:01:27.975Z
642b53bf48f67b6f21d3a357
/avatars/d5b6e908a9aba43b813d5815fdf57f9e.svg
Hani Itani
user
itanh0b
6813376e2379095ee572f2fa
Bernard Ghanem
admin_assigned
2025-05-05T08:01:34.030Z
6808bf97ffadd78ec71cb721
/avatars/9adca3142c06b8f69889fcbe85fa374d.svg
Bernard Ghanem
user
bernardghanem
Beyond the Last Answer: Your Reasoning Trace Uncovers More than You
  Think
Large Language Models (LLMs) leverage step-by-step reasoning to solve complex
problems. Standard evaluation practice involves generating a complete reasoning
trace and assessing the correctness of the final answer presented at its
conclusion. In this paper, we challenge the reliance on the final answer by
posing the following two questions: Does the final answer reliably represent
the model's optimal conclusion? Can alternative reasoning paths yield different
results? To answer these questions, we analyze intermediate reasoning steps,
termed subthoughts, and propose a method based on our findings. Our approach
involves segmenting a reasoning trace into sequential subthoughts based on
linguistic cues. We start by prompting the model to generate continuations from
the end-point of each intermediate subthought. We extract a potential answer
from every completed continuation originating from different subthoughts. We
find that aggregating these answers by selecting the most frequent one (the
mode) often yields significantly higher accuracy compared to relying solely on
the answer derived from the original complete trace. Analyzing the consistency
among the answers derived from different subthoughts reveals characteristics
that correlate with the model's confidence and correctness, suggesting
potential for identifying less reliable answers. Our experiments across various
LLMs and challenging mathematical reasoning datasets (AIME2024 and AIME2025)
show consistent accuracy improvements, with gains reaching up to 13\% and 10\%
respectively. Implementation is available at:
https://github.com/hammoudhasan/SubthoughtReasoner.
Aggregating answers from multiple subthoughts in reasoning traces improves the accuracy of Large Language Models beyond relying on the final answer.
Large Language Models step-by-step reasoning reasoning trace subthoughts linguistic cues answer aggregation mode selection model confidence AIME2024 AIME2025