Chain-of-Thought Tokens are Computer Program Variables
Chain-of-thoughts (CoT) requires large language models (LLMs) to generate
intermediate steps before reaching the final answer, and has been proven
effective to help LLMs solve complex reasoning tasks. However, the inner
mechanism of CoT still remains largely unclear. In this paper, we empirically
study the role of CoT tokens in LLMs on two compositional tasks: multi-digit
multiplication and dynamic programming. While CoT is essential for solving
these problems, we find that preserving only tokens that store intermediate
results would achieve comparable performance. Furthermore, we observe that
storing intermediate results in an alternative latent form will not affect
model performance. We also randomly intervene some values in CoT, and notice
that subsequent CoT tokens and the final answer would change correspondingly.
These findings suggest that CoT tokens may function like variables in computer
programs but with potential drawbacks like unintended shortcuts and
computational complexity limits between tokens. The code and data are available
at https://github.com/solitaryzero/CoTs_are_Variables.
681dc1b7965798cccfeab83c
Fangwei Zhu
admin_assigned
2025-05-09T09:07:17.372Z
654cca3fe1b4cd6d40d5a7ae
/avatars/4d09531277e16ad71474fc888c16b227.svg
Fangwei Zhu
user
soliz1998
681dc1b7965798cccfeab83d
Peiyi Wang
admin_assigned
2025-05-09T09:07:23.247Z
656873f33fd0bf1f82558695
/avatars/7a085da2e2a91d7f41988501a573ebf9.svg
PEIYI, WANG
user
peiyiwang89
681dc1b7965798cccfeab83e
Zhifang Sui
Chain-of-Thought Tokens are Computer Program Variables
Chain-of-thoughts (CoT) requires large language models (LLMs) to generate
intermediate steps before reaching the final answer, and has been proven
effective to help LLMs solve complex reasoning tasks. However, the inner
mechanism of CoT still remains largely unclear. In this paper, we empirically
study the role of CoT tokens in LLMs on two compositional tasks: multi-digit
multiplication and dynamic programming. While CoT is essential for solving
these problems, we find that preserving only tokens that store intermediate
results would achieve comparable performance. Furthermore, we observe that
storing intermediate results in an alternative latent form will not affect
model performance. We also randomly intervene some values in CoT, and notice
that subsequent CoT tokens and the final answer would change correspondingly.
These findings suggest that CoT tokens may function like variables in computer
programs but with potential drawbacks like unintended shortcuts and
computational complexity limits between tokens. The code and data are available
at https://github.com/solitaryzero/CoTs_are_Variables.
Intermediate result tokens in chain-of-thought (CoT) processes of large language models are crucial for solving complex tasks and function similarly to variables, but can be susceptible to unintended shortcuts and complexity issues.
chain-of-thought LLMs compositional tasks multi-digit multiplication dynamic programming intermediate result tokens latent form unintended shortcuts computational complexity