Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
Chain-of-Thought (CoT) prompting has been shown to improve Large Language
Model (LLM) performance on various tasks. With this approach, LLMs appear to
produce human-like reasoning steps before providing answers (a.k.a., CoT
reasoning), which often leads to the perception that they engage in deliberate
inferential processes. However, some initial findings suggest that CoT
reasoning may be more superficial than it appears, motivating us to explore
further. In this paper, we study CoT reasoning via a data distribution lens and
investigate if CoT reasoning reflects a structured inductive bias learned from
in-distribution data, allowing the model to conditionally generate reasoning
paths that approximate those seen during training. Thus, its effectiveness is
fundamentally bounded by the degree of distribution discrepancy between the
training data and the test queries. With this lens, we dissect CoT reasoning
via three dimensions: task, length, and format. To investigate each dimension,
we design DataAlchemy, an isolated and controlled environment to train LLMs
from scratch and systematically probe them under various distribution
conditions. Our results reveal that CoT reasoning is a brittle mirage that
vanishes when it is pushed beyond training distributions. This work offers a
deeper understanding of why and when CoT reasoning fails, emphasizing the
ongoing challenge of achieving genuine and generalizable reasoning.
689414b0741a16f544fbcec8
Chengshuai Zhao
claimed_verified
2025-08-07T10:38:44.551Z
65b2fae679954e21ac426aec
https://cdn-avatars.huggingface.co/v1/production/uploads/65b2fae679954e21ac426aec/LybSb_awygRTQinm1npUq.jpeg
Chengshuai Zhao
user
chengshuaizhao
689414b0741a16f544fbcec9
Zhen Tan
claimed_verified
2025-08-08T16:22:06.855Z
65b2fabb14031ba201cb5aa8
/avatars/a5880006c9f00de84e779497e3eb84a5.svg
Zhen Tan
user
ztan36
689414b0741a16f544fbceca
Pingchuan Ma
689414b0741a16f544fbcecb
Dawei Li
claimed_verified
2025-08-07T10:38:54.238Z
6474e1afb68461d5cf7c41cc
https://cdn-avatars.huggingface.co/v1/production/uploads/6474e1afb68461d5cf7c41cc/bcoiD_qPrjHUBlB259djg.png
Dawei Li
user
wjldw
689414b0741a16f544fbcecc
Bohan Jiang
claimed_verified
2025-08-07T10:38:50.992Z
61f087a0a57920a251ec1a6f
/avatars/4402b7986152bb37e02f1305c6bcce2e.svg
Bohan Jiang
user
Bohan-Jiang
689414b0741a16f544fbcecd
Yancheng Wang
claimed_verified
2025-08-11T06:49:42.671Z
669002cc57188a0640cc205d
/avatars/7da17b2d9f9c69f231f36025b59098ca.svg
Yancheng Wang
user
peterycw
689414b0741a16f544fbcece
Yingzhen Yang
689414b0741a16f544fbcecf
Huan Liu
Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
Chain-of-Thought (CoT) prompting has been shown to improve Large Language
Model (LLM) performance on various tasks. With this approach, LLMs appear to
produce human-like reasoning steps before providing answers (a.k.a., CoT
reasoning), which often leads to the perception that they engage in deliberate
inferential processes. However, some initial findings suggest that CoT
reasoning may be more superficial than it appears, motivating us to explore
further. In this paper, we study CoT reasoning via a data distribution lens and
investigate if CoT reasoning reflects a structured inductive bias learned from
in-distribution data, allowing the model to conditionally generate reasoning
paths that approximate those seen during training. Thus, its effectiveness is
fundamentally bounded by the degree of distribution discrepancy between the
training data and the test queries. With this lens, we dissect CoT reasoning
via three dimensions: task, length, and format. To investigate each dimension,
we design DataAlchemy, an isolated and controlled environment to train LLMs
from scratch and systematically probe them under various distribution
conditions. Our results reveal that CoT reasoning is a brittle mirage that
vanishes when it is pushed beyond training distributions. This work offers a
deeper understanding of why and when CoT reasoning fails, emphasizing the
ongoing challenge of achieving genuine and generalizable reasoning.
CoT reasoning in LLMs is found to be limited by the distribution discrepancy between training and test data, suggesting it is not a robust form of reasoning.
Chain-of-Thought Large Language Model CoT reasoning inductive bias DataAlchemy distribution discrepancy reasoning paths generalizable reasoning