PEAR: Phase Entropy Aware Reward for Efficient Reasoning
Large Reasoning Models (LRMs) have achieved impressive performance on complex
reasoning tasks by generating detailed chain-of-thought (CoT) explanations.
However, these responses are often excessively long, containing redundant
reasoning steps that inflate inference cost and reduce usability. Controlling
the length of generated reasoning without sacrificing accuracy remains an open
challenge. Through a systematic empirical analysis, we reveal a consistent
positive correlation between model entropy and response length at different
reasoning stages across diverse LRMs: the thinking phase exhibits higher
entropy, reflecting exploratory behavior of longer responses, while the final
answer phase shows lower entropy, indicating a more deterministic solution.
This observation suggests that entropy at different reasoning stages can serve
as a control knob for balancing conciseness and performance. Based on this
insight, this paper introduces Phase Entropy Aware Reward (PEAR), a reward
mechanism that incorporating phase-dependent entropy into the reward design.
Instead of treating all tokens uniformly, PEAR penalize excessive entropy
during the thinking phase and allowing moderate exploration at the final answer
phase, which encourages models to generate concise reasoning traces that retain
sufficient flexibility to solve the task correctly. This enables adaptive
control of response length without relying on explicit length targets or rigid
truncation rules. Extensive experiments across four benchmarks demonstrate that
PEAR consistently reduces response length while sustaining competitive accuracy
across model scales. In addition, PEAR demonstrates strong out-of-distribution
(OOD) robustness beyond the training distribution. Our code is available at:
https://github.com/iNLP-Lab/PEAR.
68eda813de1fee572713a6ad
Chen Huang
claimed_verified
2025-10-14T07:31:17.491Z
65d7b983baa72790a1151923
/avatars/938531e84ca01a0c5a2a174057e3e9c5.svg
Chen Huang
user
Albus-Chen
68eda813de1fee572713a6ae
Wei Lu
68eda813de1fee572713a6af
Wenxuan Zhang
PEAR: Phase Entropy Aware Reward for Efficient Reasoning
Large Reasoning Models (LRMs) have achieved impressive performance on complex
reasoning tasks by generating detailed chain-of-thought (CoT) explanations.
However, these responses are often excessively long, containing redundant
reasoning steps that inflate inference cost and reduce usability. Controlling
the length of generated reasoning without sacrificing accuracy remains an open
challenge. Through a systematic empirical analysis, we reveal a consistent
positive correlation between model entropy and response length at different
reasoning stages across diverse LRMs: the thinking phase exhibits higher
entropy, reflecting exploratory behavior of longer responses, while the final
answer phase shows lower entropy, indicating a more deterministic solution.
This observation suggests that entropy at different reasoning stages can serve
as a control knob for balancing conciseness and performance. Based on this
insight, this paper introduces Phase Entropy Aware Reward (PEAR), a reward
mechanism that incorporating phase-dependent entropy into the reward design.
Instead of treating all tokens uniformly, PEAR penalize excessive entropy
during the thinking phase and allowing moderate exploration at the final answer
phase, which encourages models to generate concise reasoning traces that retain
sufficient flexibility to solve the task correctly. This enables adaptive
control of response length without relying on explicit length targets or rigid
truncation rules. Extensive experiments across four benchmarks demonstrate that
PEAR consistently reduces response length while sustaining competitive accuracy
across model scales. In addition, PEAR demonstrates strong out-of-distribution
(OOD) robustness beyond the training distribution. Our code is available at:
https://github.com/iNLP-Lab/PEAR.
A reward mechanism called Phase Entropy Aware Reward (PEAR) controls the length of reasoning in large models by adjusting entropy at different stages, balancing conciseness and accuracy.
Large Reasoning Models chain-of-thought response length model entropy thinking phase final answer phase Phase Entropy Aware Reward PEAR out-of-distribution robustness