ARM: Adaptive Reasoning Model
While large reasoning models demonstrate strong performance on complex tasks,
they lack the ability to adjust reasoning token usage based on task difficulty.
This often leads to the "overthinking" problem -- excessive and unnecessary
reasoning -- which, although potentially mitigated by human intervention to
control the token budget, still fundamentally contradicts the goal of achieving
fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a
reasoning model capable of adaptively selecting appropriate reasoning formats
based on the task at hand. These formats include three efficient ones -- Direct
Answer, Short CoT, and Code -- as well as a more elaborate format, Long CoT. To
train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy
Optimization (GRPO), which addresses the format collapse issue in traditional
GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by
an average of 30%, and up to 70%, while maintaining performance comparable to
the model that relies solely on Long CoT. Furthermore, not only does it improve
inference efficiency through reduced token generation, but it also brings a 2x
speedup in training. In addition to the default Adaptive Mode, ARM supports two
additional reasoning modes: 1) Instruction-Guided Mode, which allows users to
explicitly specify the reasoning format via special tokens -- ideal when the
appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode,
which aggregates the outputs of the three efficient formats and resorts to Long
CoT in case of disagreement, prioritizing performance with higher token usage.
68352b5803548b71276c1a6f
Siye Wu
admin_assigned
2025-05-27T08:25:16.627Z
64f2a228f40f35cfa3e8edfd
/avatars/0671cb4df8f3d3bcaaa95aad3d0a46c2.svg
Siye Wu
user
Siye01
68352b5803548b71276c1a70
Jian Xie
claimed_verified
2025-05-27T07:50:53.006Z
62d65139667051e0a29bffe7
/avatars/0252aa2bcd4cf1c8e4b87e5f164b6da5.svg
Jian Xie
user
hsaest
68352b5803548b71276c1a71
Yikai Zhang
admin_assigned
2025-05-27T08:25:41.797Z
63e8b792ca4fc7d30de6975b
/avatars/57237f54d61d479df15209497a3f531e.svg
Yikai Zhang
user
Arist12
68352b5803548b71276c1a72
Aili Chen
claimed_verified
2025-05-29T20:18:44.972Z
63f86b099f87cc3e645b51d9
/avatars/27ca5ba425640bf67474cee871e8e53a.svg
Ellie Chen
user
sheep33333
68352b5803548b71276c1a73
Kai Zhang
68352b5803548b71276c1a74
Yu Su
68352b5803548b71276c1a75
Yanghua Xiao
ARM: Adaptive Reasoning Model
While large reasoning models demonstrate strong performance on complex tasks,
they lack the ability to adjust reasoning token usage based on task difficulty.
This often leads to the "overthinking" problem -- excessive and unnecessary
reasoning -- which, although potentially mitigated by human intervention to
control the token budget, still fundamentally contradicts the goal of achieving
fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a
reasoning model capable of adaptively selecting appropriate reasoning formats
based on the task at hand. These formats include three efficient ones -- Direct
Answer, Short CoT, and Code -- as well as a more elaborate format, Long CoT. To
train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy
Optimization (GRPO), which addresses the format collapse issue in traditional
GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by
an average of 30%, and up to 70%, while maintaining performance comparable to
the model that relies solely on Long CoT. Furthermore, not only does it improve
inference efficiency through reduced token generation, but it also brings a 2x
speedup in training. In addition to the default Adaptive Mode, ARM supports two
additional reasoning modes: 1) Instruction-Guided Mode, which allows users to
explicitly specify the reasoning format via special tokens -- ideal when the
appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode,
which aggregates the outputs of the three efficient formats and resorts to Long
CoT in case of disagreement, prioritizing performance with higher token usage.
Adaptive Reasoning Model (ARM) uses Ada-GRPO to reduce token usage and improve efficiency across different reasoning modes.
Adaptive Reasoning Model ARM Ada-GRPO Group Relative Policy Optimization GRPO format collapse token efficiency inference efficiency Direct Answer Short CoT Code Long CoT Adaptive Mode Instruction-Guided Mode Consensus-Guided Mode