Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs
  at Test Time
Test-Time Scaling (TTS) enhances the reasoning ability of large language
models (LLMs) by allocating additional computation during inference. However,
existing approaches primarily rely on output-level sampling while overlooking
the role of model architecture. In mainstream Mixture-of-Experts (MoE) LLMs, we
observe that varying the number of activated experts yields complementary
solution sets with stable accuracy, revealing a new and underexplored source of
diversity. Motivated by this observation, we propose Dynamic Experts Search
(DES), a TTS strategy that elevates expert activation into a controllable
dimension of the search space. DES integrates two key components: (1) Dynamic
MoE, which enables direct control of expert counts during inference to generate
diverse reasoning trajectories without additional cost; and (2) Expert
Configuration Inheritance, which preserves consistent expert counts within a
reasoning path while varying them across runs, thereby balancing stability and
diversity throughout the search. Extensive experiments across MoE
architectures, verifiers and reasoning benchmarks (i.e., math, code and
knowledge) demonstrate that DES reliably outperforms TTS baselines, enhancing
accuracy and stability without additional cost. These results highlight DES as
a practical and scalable form of architecture-aware TTS, illustrating how
structural flexibility in modern LLMs can advance reasoning.
68db4efed2bf1f4b15ec7497
Yixuan Han
claimed_verified
2025-09-30T07:12:25.187Z
673d4716cc1ef74a349cd2ad
/avatars/a88f1d461c199a2caa1d5e13b70921fe.svg
Yixuan Han
user
yixuan7878
68db4efed2bf1f4b15ec7498
Fan Ma
68db4efed2bf1f4b15ec7499
Ruijie Quan
68db4efed2bf1f4b15ec749a
Yi Yang
Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs
  at Test Time
Test-Time Scaling (TTS) enhances the reasoning ability of large language
models (LLMs) by allocating additional computation during inference. However,
existing approaches primarily rely on output-level sampling while overlooking
the role of model architecture. In mainstream Mixture-of-Experts (MoE) LLMs, we
observe that varying the number of activated experts yields complementary
solution sets with stable accuracy, revealing a new and underexplored source of
diversity. Motivated by this observation, we propose Dynamic Experts Search
(DES), a TTS strategy that elevates expert activation into a controllable
dimension of the search space. DES integrates two key components: (1) Dynamic
MoE, which enables direct control of expert counts during inference to generate
diverse reasoning trajectories without additional cost; and (2) Expert
Configuration Inheritance, which preserves consistent expert counts within a
reasoning path while varying them across runs, thereby balancing stability and
diversity throughout the search. Extensive experiments across MoE
architectures, verifiers and reasoning benchmarks (i.e., math, code and
knowledge) demonstrate that DES reliably outperforms TTS baselines, enhancing
accuracy and stability without additional cost. These results highlight DES as
a practical and scalable form of architecture-aware TTS, illustrating how
structural flexibility in modern LLMs can advance reasoning.
Dynamic Experts Search (DES) enhances large language models by controlling expert activation during inference, improving accuracy and stability without additional cost.
Mixture-of-Experts (MoE) Dynamic Experts Search (DES) Dynamic MoE Expert Configuration Inheritance reasoning trajectories verifiers reasoning benchmarks math code knowledge