DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning
  Trajectories Search
Enhancing the capability of large language models (LLMs) in reasoning has
gained significant attention in recent years. Previous studies have
demonstrated the effectiveness of various prompting strategies in aiding LLMs
in reasoning (called "reasoning actions"), such as step-by-step thinking,
reflecting before answering, solving with programs, and their combinations.
However, these approaches often applied static, predefined reasoning actions
uniformly to all questions, without considering the specific characteristics of
each question or the capability of the task-solving LLM. In this paper, we
propose DOTS, an approach enabling LLMs to reason dynamically via optimal
reasoning trajectory search, tailored to the specific characteristics of each
question and the inherent capability of the task-solving LLM. Our approach
involves three key steps: i) defining atomic reasoning action modules that can
be composed into various reasoning action trajectories; ii) searching for the
optimal action trajectory for each training question through iterative
exploration and evaluation for the specific task-solving LLM; and iii) using
the collected optimal trajectories to train an LLM to plan for the reasoning
trajectories of unseen questions. In particular, we propose two learning
paradigms, i.e., fine-tuning an external LLM as a planner to guide the
task-solving LLM, or directly fine-tuning the task-solving LLM with an
internalized capability for reasoning actions planning. Our experiments across
eight reasoning tasks show that our method consistently outperforms static
reasoning techniques and the vanilla instruction tuning approach. Further
analysis reveals that our method enables LLMs to adjust their computation based
on problem complexity, allocating deeper thinking and reasoning to harder
problems.
6706c8cc10d2faa686333496
Murong Yue
claimed_verified
2024-10-28T11:25:50.840Z
63e1ac9f7fbb6ae4d4f312a3
/avatars/7d999543d3ca3df3fc8bb96cd3a3f8cb.svg
Murong Yue
user
MurongYue
6706c8cc10d2faa686333497
Wenlin Yao
6706c8cc10d2faa686333498
Haitao Mi
6706c8cc10d2faa686333499
Dian Yu
6706c8cc10d2faa68633349a
Ziyu Yao
claimed_verified
2025-05-27T15:50:35.621Z
62d87e143cd901df9446a37a
/avatars/b9a7d24bbe6e49ac90902fafee3a74e7.svg
Ziyu Yao
user
ziyuyao
6706c8cc10d2faa68633349b
Dong Yu
DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning
  Trajectories Search
Enhancing the capability of large language models (LLMs) in reasoning has
gained significant attention in recent years. Previous studies have
demonstrated the effectiveness of various prompting strategies in aiding LLMs
in reasoning (called "reasoning actions"), such as step-by-step thinking,
reflecting before answering, solving with programs, and their combinations.
However, these approaches often applied static, predefined reasoning actions
uniformly to all questions, without considering the specific characteristics of
each question or the capability of the task-solving LLM. In this paper, we
propose DOTS, an approach enabling LLMs to reason dynamically via optimal
reasoning trajectory search, tailored to the specific characteristics of each
question and the inherent capability of the task-solving LLM. Our approach
involves three key steps: i) defining atomic reasoning action modules that can
be composed into various reasoning action trajectories; ii) searching for the
optimal action trajectory for each training question through iterative
exploration and evaluation for the specific task-solving LLM; and iii) using
the collected optimal trajectories to train an LLM to plan for the reasoning
trajectories of unseen questions. In particular, we propose two learning
paradigms, i.e., fine-tuning an external LLM as a planner to guide the
task-solving LLM, or directly fine-tuning the task-solving LLM with an
internalized capability for reasoning actions planning. Our experiments across
eight reasoning tasks show that our method consistently outperforms static
reasoning techniques and the vanilla instruction tuning approach. Further
analysis reveals that our method enables LLMs to adjust their computation based
on problem complexity, allocating deeper thinking and reasoning to harder
problems.
An approach called DOTS enables large language models to dynamically reason by selecting optimal reasoning trajectories tailored to each question, improving performance across various reasoning tasks.
large language models LLMs reasoning actions step-by-step thinking reflecting solving with programs dynamic reasoning optimal reasoning trajectory search atomic reasoning action modules training question reasoning action trajectories iterative exploration evaluation fine-tuning external LLM internalized capability reasoning actions planning problem complexity deeper thinking