ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level
  Entropy Shaping
Recent advances in multimodal large reasoning models (MLRMs) have
substantially improved their ability to solve complex textual and visual tasks.
However, these models tend to overthink on simple problems, producing
unnecessarily lengthy reasoning traces, while under-exploring on challenging
ones, leading to missed solutions. To address this imbalance, we propose ARES,
a unified open-source framework for adaptive reasoning that dynamically
allocates exploration effort based on task difficulty. Our approach is
motivated by two key empirical findings: (i) while single-token entropy is
noisy, high window-entropy (HWE) tokens (token-level entropies averaged under a
sliding window) can reliably capture reasoning-critical moments; and (ii)
reducing HWE usage benefits easy problems, while increasing it is essential for
solving hard ones. Building on these insights, ARES introduces a two-stage
training pipeline. In the Adaptive Cold-Start stage, we curate multimodal and
textual data paired with reasoning traces of length proportional to problem
difficulty, equipping the model with initial difficulty awareness. In the
second stage, we develop Adaptive Entropy Policy Optimization (AEPO), which
uses HWE tokens as exploration triggers to decide when to explore, and a
hierarchical entropy reward with dynamic KL control to decide how much to
explore. Extensive experiments demonstrate that ARES achieves superior
performance and reasoning efficiency across diverse mathematical, logical, and
multimodal benchmarks, while closing the gap to leading commercial systems
under significantly lower inference costs.
68ebaa94012880a10864bbee
Shuang Chen
claimed_verified
2025-10-13T10:06:21.942Z
66cc19ad6f8945277c39cd86
/avatars/f031e77d7fb9dbebbc00fba9b5dd5357.svg
Shawn
user
csfufu
68ebaa94012880a10864bbef
Yue Guo
68ebaa94012880a10864bbf0
Yimeng Ye
68ebaa94012880a10864bbf1
Shijue Huang
claimed_verified
2025-10-13T10:06:24.272Z
64ce05c631c655ff8a2e183c
/avatars/f2de7f8a1348b05f46946085e3e9718e.svg
Shijue Huang
user
JoeYing
68ebaa94012880a10864bbf2
Wenbo Hu
68ebaa94012880a10864bbf3
Haoxi Li
68ebaa94012880a10864bbf4
Manyuan Zhang
68ebaa94012880a10864bbf5
Jiayu Chen
68ebaa94012880a10864bbf6
Song Guo
68ebaa94012880a10864bbf7
Nanyun Peng
ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level
  Entropy Shaping
Recent advances in multimodal large reasoning models (MLRMs) have
substantially improved their ability to solve complex textual and visual tasks.
However, these models tend to overthink on simple problems, producing
unnecessarily lengthy reasoning traces, while under-exploring on challenging
ones, leading to missed solutions. To address this imbalance, we propose ARES,
a unified open-source framework for adaptive reasoning that dynamically
allocates exploration effort based on task difficulty. Our approach is
motivated by two key empirical findings: (i) while single-token entropy is
noisy, high window-entropy (HWE) tokens (token-level entropies averaged under a
sliding window) can reliably capture reasoning-critical moments; and (ii)
reducing HWE usage benefits easy problems, while increasing it is essential for
solving hard ones. Building on these insights, ARES introduces a two-stage
training pipeline. In the Adaptive Cold-Start stage, we curate multimodal and
textual data paired with reasoning traces of length proportional to problem
difficulty, equipping the model with initial difficulty awareness. In the
second stage, we develop Adaptive Entropy Policy Optimization (AEPO), which
uses HWE tokens as exploration triggers to decide when to explore, and a
hierarchical entropy reward with dynamic KL control to decide how much to
explore. Extensive experiments demonstrate that ARES achieves superior
performance and reasoning efficiency across diverse mathematical, logical, and
multimodal benchmarks, while closing the gap to leading commercial systems
under significantly lower inference costs.
ARES, a unified framework for adaptive reasoning, dynamically adjusts exploration effort based on task difficulty using high window-entropy tokens and hierarchical entropy rewards, improving performance and efficiency across various benchmarks.
multimodal large reasoning models MLRMs adaptive reasoning high window-entropy HWE Adaptive Cold-Start Adaptive Entropy Policy Optimization AEPO entropy reward dynamic KL control