Reasoning Models Better Express Their Confidence
Despite their strengths, large language models (LLMs) often fail to
communicate their confidence accurately, making it difficult to assess when
they might be wrong and limiting their reliability. In this work, we
demonstrate that reasoning models-LLMs that engage in extended chain-of-thought
(CoT) reasoning-exhibit superior performance not only in problem-solving but
also in accurately expressing their confidence. Specifically, we benchmark six
reasoning models across six datasets and find that they achieve strictly better
confidence calibration than their non-reasoning counterparts in 33 out of the
36 settings. Our detailed analysis reveals that these gains in calibration stem
from the slow thinking behaviors of reasoning models-such as exploring
alternative approaches and backtracking-which enable them to adjust their
confidence dynamically throughout their CoT, making it progressively more
accurate. In particular, we find that reasoning models become increasingly
better calibrated as their CoT unfolds, a trend not observed in non-reasoning
models. Moreover, removing slow thinking behaviors from the CoT leads to a
significant drop in calibration. Lastly, we show that these gains are not
exclusive to reasoning models-non-reasoning models also benefit when guided to
perform slow thinking via in-context learning.
682d55ecea67e90811b09b6b
Dongkeun Yoon
claimed_verified
2025-05-21T08:40:23.897Z
617f679fb15f8a665f3999fc
https://cdn-avatars.huggingface.co/v1/production/uploads/617f679fb15f8a665f3999fc/NW1vkLsGAlWpAQYTux05X.jpeg
Dongkeun Yoon
user
DKYoon
682d55ecea67e90811b09b6c
Seungone Kim
admin_assigned
2025-05-21T10:15:44.937Z
6469949654873f0043b09c22
https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/Lk7IJAR16Wa_sGJ2g81AQ.jpeg
Seungone Kim
user
seungone
682d55ecea67e90811b09b6d
Sohee Yang
admin_assigned
2025-05-21T10:15:51.388Z
606ae0adb579bb0e88515311
https://cdn-avatars.huggingface.co/v1/production/uploads/1617617060380-noauth.jpeg
Sohee Yang
user
soheeyang
682d55ecea67e90811b09b6e
Sunkyoung Kim
admin_assigned
2025-05-21T10:16:00.139Z
628efeff6d8dcc7ab5263881
/avatars/1e894ccd7ba206a755b0b9af9f22ead1.svg
Sunkyoung Kim
user
Sunkyoung
682d55ecea67e90811b09b6f
Soyeon Kim
682d55ecea67e90811b09b70
Yongil Kim
admin_assigned
2025-05-21T10:17:46.138Z
66a9c79cf59bc4c77baac0d5
/avatars/57e9943b339a5b9be21b411e5286821c.svg
Yongil Kim
user
YongilKim
682d55ecea67e90811b09b71
Eunbi Choi
admin_assigned
2025-05-21T10:17:29.667Z
6044fd39e6aa3e130cb92867
https://cdn-avatars.huggingface.co/v1/production/uploads/6044fd39e6aa3e130cb92867/L5hb8vpHY6SKMEL-Xacma.jpeg
Eunbi Choi
user
unbiarirang
682d55ecea67e90811b09b72
Yireun Kim
extracted_confirmed
2025-05-21T04:27:50.114Z
660260cf1737e5cd4a826550
https://cdn-avatars.huggingface.co/v1/production/uploads/660260cf1737e5cd4a826550/AlSfoM2WtqPjLtYR6x7Wf.jpeg
Yireun Kim
user
yireun
682d55ecea67e90811b09b73
Minjoon Seo
admin_assigned
2025-05-21T10:17:22.628Z
621f05ba970615ad5861ceb1
/avatars/7e1902aa71369a524afda9b0a9e88e22.svg
Minjoon Seo
user
minjoon
Reasoning Models Better Express Their Confidence
Despite their strengths, large language models (LLMs) often fail to
communicate their confidence accurately, making it difficult to assess when
they might be wrong and limiting their reliability. In this work, we
demonstrate that reasoning models-LLMs that engage in extended chain-of-thought
(CoT) reasoning-exhibit superior performance not only in problem-solving but
also in accurately expressing their confidence. Specifically, we benchmark six
reasoning models across six datasets and find that they achieve strictly better
confidence calibration than their non-reasoning counterparts in 33 out of the
36 settings. Our detailed analysis reveals that these gains in calibration stem
from the slow thinking behaviors of reasoning models-such as exploring
alternative approaches and backtracking-which enable them to adjust their
confidence dynamically throughout their CoT, making it progressively more
accurate. In particular, we find that reasoning models become increasingly
better calibrated as their CoT unfolds, a trend not observed in non-reasoning
models. Moreover, removing slow thinking behaviors from the CoT leads to a
significant drop in calibration. Lastly, we show that these gains are not
exclusive to reasoning models-non-reasoning models also benefit when guided to
perform slow thinking via in-context learning.
Reasoning models, which perform chain-of-thought reasoning, exhibit superior confidence calibration compared to non-reasoning models by dynamically adjusting their confidence during the reasoning process.
large language models LLMs chain-of-thought CoT reasoning confidence calibration benchmarking datasets alternative approaches backtracking in-context learning