Rewiring Experts on the Fly:Continuous Rerouting for Better Online
  Adaptation in Mixture-of-Expert models
Mixture-of-Experts (MoE) models achieve efficient scaling through sparse
expert activation, but often suffer from suboptimal routing decisions due to
distribution shifts in deployment. While existing test-time adaptation methods
could potentially address these issues, they primarily focus on dense models
and require access to external data, limiting their practical applicability to
MoE architectures. However, we find that, instead of relying on reference data,
we can optimize MoE expert selection on-the-fly based only on input context. As
such, we propose a data-free, online test-time framework that
continuously adapts MoE routing decisions during text generation without
external supervision or data. Our method cycles between two phases: During the
prefill stage, and later in regular intervals, we optimize the routing
decisions of the model using self-supervision based on the already generated
sequence. Then, we generate text as normal, maintaining the modified router
until the next adaption. We implement this through lightweight additive vectors
that only update router logits in selected layers, maintaining computational
efficiency while preventing over-adaptation. The experimental results show
consistent performance gains on challenging reasoning tasks while maintaining
robustness to context shifts. For example, our method achieves a 5.5\%
improvement on HumanEval with OLMoE. Furthermore, owing to its plug-and-play
property, our method naturally complements existing test-time scaling
techniques, e.g., achieving 6\% average gains when incorporated with
self-consistency on DeepSeek-V2-Lite.
68f5efbd8589920bf4d32247
Guinan Su
claimed_verified
2025-10-21T14:04:38.633Z
630ed6cfc9af0163b96a71aa
/avatars/bf984bb1c2f68336bcf042c6fc7e0d9d.svg
Guinan-Su
user
guinansu
68f5efbd8589920bf4d32248
Yanwu Yang
68f5efbd8589920bf4d32249
Li Shen
68f5efbd8589920bf4d3224a
Lu Yin
68f5efbd8589920bf4d3224b
Shiwei Liu
68f5efbd8589920bf4d3224c
Jonas Geiping
Rewiring Experts on the Fly:Continuous Rerouting for Better Online
  Adaptation in Mixture-of-Expert models
Mixture-of-Experts (MoE) models achieve efficient scaling through sparse
expert activation, but often suffer from suboptimal routing decisions due to
distribution shifts in deployment. While existing test-time adaptation methods
could potentially address these issues, they primarily focus on dense models
and require access to external data, limiting their practical applicability to
MoE architectures. However, we find that, instead of relying on reference data,
we can optimize MoE expert selection on-the-fly based only on input context. As
such, we propose a data-free, online test-time framework that
continuously adapts MoE routing decisions during text generation without
external supervision or data. Our method cycles between two phases: During the
prefill stage, and later in regular intervals, we optimize the routing
decisions of the model using self-supervision based on the already generated
sequence. Then, we generate text as normal, maintaining the modified router
until the next adaption. We implement this through lightweight additive vectors
that only update router logits in selected layers, maintaining computational
efficiency while preventing over-adaptation. The experimental results show
consistent performance gains on challenging reasoning tasks while maintaining
robustness to context shifts. For example, our method achieves a 5.5\%
improvement on HumanEval with OLMoE. Furthermore, owing to its plug-and-play
property, our method naturally complements existing test-time scaling
techniques, e.g., achieving 6\% average gains when incorporated with
self-consistency on DeepSeek-V2-Lite.
A data-free, online test-time framework optimizes MoE routing decisions during text generation using self-supervision, improving performance and robustness without external data.
Mixture-of-Experts MoE sparse expert activation test-time adaptation self-supervision router logits prefill stage context shifts HumanEval OLMoE self-consistency DeepSeek-V2-Lite