From Token to Action: State Machine Reasoning to Mitigate Overthinking
  in Information Retrieval
Chain-of-Thought (CoT) prompting enables complex reasoning in large language
models (LLMs), including applications in information retrieval (IR). However,
it often leads to overthinking, where models produce excessively long and
semantically redundant traces with little or no benefit. We identify two key
challenges in IR: redundant trajectories that revisit similar states and
misguided reasoning that diverges from user intent. To address these, we
propose State Machine Reasoning (SMR), a transition-based reasoning framework
composed of discrete actions (Refine, Rerank, Stop) that support early stopping
and fine-grained control. Experiments on the BEIR and BRIGHT benchmarks show
that SMR improves retrieval performance (nDCG@10) by 3.4% while reducing token
usage by 74.4%. It generalizes across LLMs and retrievers without requiring
task-specific tuning, offering a practical alternative to conventional CoT
reasoning. The code and details are available at https://github.com/ldilab/SMR.
683b4b86c4b9677f3f7125e5
Dohyeon Lee
claimed_verified
2025-06-03T08:50:07.024Z
6683c3b04905815dcffe7a21
/avatars/73c50843d99b6fb8b1ee8fe11106c4ce.svg
Dohyeon Lee
user
waylight3
683b4b86c4b9677f3f7125e6
Yeonseok Jeong
claimed_verified
2025-06-02T07:42:20.758Z
645aedd221ab438e732bff43
/avatars/31e14fee670fd5ddd296ea0249dbf710.svg
Yeonseok Jeong
user
yeonseokjeong
683b4b86c4b9677f3f7125e7
Seung-won Hwang
From Token to Action: State Machine Reasoning to Mitigate Overthinking
  in Information Retrieval
Chain-of-Thought (CoT) prompting enables complex reasoning in large language
models (LLMs), including applications in information retrieval (IR). However,
it often leads to overthinking, where models produce excessively long and
semantically redundant traces with little or no benefit. We identify two key
challenges in IR: redundant trajectories that revisit similar states and
misguided reasoning that diverges from user intent. To address these, we
propose State Machine Reasoning (SMR), a transition-based reasoning framework
composed of discrete actions (Refine, Rerank, Stop) that support early stopping
and fine-grained control. Experiments on the BEIR and BRIGHT benchmarks show
that SMR improves retrieval performance (nDCG@10) by 3.4% while reducing token
usage by 74.4%. It generalizes across LLMs and retrievers without requiring
task-specific tuning, offering a practical alternative to conventional CoT
reasoning. The code and details are available at https://github.com/ldilab/SMR.
State Machine Reasoning (SMR) improves information retrieval performance and reduces token usage in large language models by addressing overthinking through a discrete action framework.
Chain-of-Thought State Machine Reasoning IR redundant trajectories misguided reasoning early stopping nDCG@10