Implicit Reasoning in Transformers is Reasoning through Shortcuts
Test-time compute is emerging as a new paradigm for enhancing language
models' complex multi-step reasoning capabilities, as demonstrated by the
success of OpenAI's o1 and o3, as well as DeepSeek's R1. Compared to explicit
reasoning in test-time compute, implicit reasoning is more inference-efficient,
requiring fewer generated tokens. However, why does the advanced reasoning
capability fail to emerge in the implicit reasoning style? In this work, we
train GPT-2 from scratch on a curated multi-step mathematical reasoning dataset
and conduct analytical experiments to investigate how language models perform
implicit reasoning in multi-step tasks. Our findings reveal: 1) Language models
can perform step-by-step reasoning and achieve high accuracy in both in-domain
and out-of-domain tests via implicit reasoning. However, this capability only
emerges when trained on fixed-pattern data. 2) Conversely, implicit reasoning
abilities emerging from training on unfixed-pattern data tend to overfit a
specific pattern and fail to generalize further. Notably, this limitation is
also observed in state-of-the-art large language models. These findings suggest
that language models acquire implicit reasoning through shortcut learning,
enabling strong performance on tasks with similar patterns while lacking
generalization.
67cfa4ecd8cb8688d7d6d8b5
Tianhe Lin
67cfa4ecd8cb8688d7d6d8b6
Jian Xie
claimed_verified
2025-03-12T08:42:36.765Z
62d65139667051e0a29bffe7
/avatars/0252aa2bcd4cf1c8e4b87e5f164b6da5.svg
Jian Xie
user
hsaest
67cfa4ecd8cb8688d7d6d8b7
Siyu Yuan
admin_assigned
2025-03-12T15:48:31.002Z
62d62b333bf5e059f7d2b286
https://cdn-avatars.huggingface.co/v1/production/uploads/1668513815771-62d62b333bf5e059f7d2b286.jpeg
Siyu Yuan
user
siyuyuan
67cfa4ecd8cb8688d7d6d8b8
Deqing Yang
Implicit Reasoning in Transformers is Reasoning through Shortcuts
Test-time compute is emerging as a new paradigm for enhancing language
models' complex multi-step reasoning capabilities, as demonstrated by the
success of OpenAI's o1 and o3, as well as DeepSeek's R1. Compared to explicit
reasoning in test-time compute, implicit reasoning is more inference-efficient,
requiring fewer generated tokens. However, why does the advanced reasoning
capability fail to emerge in the implicit reasoning style? In this work, we
train GPT-2 from scratch on a curated multi-step mathematical reasoning dataset
and conduct analytical experiments to investigate how language models perform
implicit reasoning in multi-step tasks. Our findings reveal: 1) Language models
can perform step-by-step reasoning and achieve high accuracy in both in-domain
and out-of-domain tests via implicit reasoning. However, this capability only
emerges when trained on fixed-pattern data. 2) Conversely, implicit reasoning
abilities emerging from training on unfixed-pattern data tend to overfit a
specific pattern and fail to generalize further. Notably, this limitation is
also observed in state-of-the-art large language models. These findings suggest
that language models acquire implicit reasoning through shortcut learning,
enabling strong performance on tasks with similar patterns while lacking
generalization.
Language models can perform implicit reasoning for multi-step tasks but overfit when trained on unfixed-pattern data, leading to poor generalization.
test-time compute implicit reasoning GPT-2 multi-step mathematical reasoning step-by-step reasoning in-domain tests out-of-domain tests fixed-pattern data unfixed-pattern data shortcut learning generalization