Dr.LLM: Dynamic Layer Routing in LLMs
Large Language Models (LLMs) process every token through all layers of a
transformer stack, causing wasted computation on simple queries and
insufficient flexibility for harder ones that need deeper reasoning.
Adaptive-depth methods can improve efficiency, but prior approaches rely on
costly inference-time search, architectural changes, or large-scale retraining,
and in practice often degrade accuracy despite efficiency gains. We introduce
Dr.LLM, Dynamic routing of Layers for LLMs, a retrofittable framework that
equips pretrained models with lightweight per-layer routers deciding to skip,
execute, or repeat a block. Routers are trained with explicit supervision:
using Monte Carlo Tree Search (MCTS), we derive high-quality layer
configurations that preserve or improve accuracy under a compute budget. Our
design, windowed pooling for stable routing, focal loss with class balancing,
and bottleneck MLP routers, ensures robustness under class imbalance and long
sequences. On ARC (logic) and DART (math), Dr.LLM improves accuracy by up to
+3.4%p while saving 5 layers per example on average. Routers generalize to
out-of-domain tasks (MMLU, GSM8k, AIME, TruthfulQA, SQuADv2, GPQA, PIQA,
AGIEval) with only 0.85% accuracy drop while retaining efficiency, and
outperform prior routing methods by up to +7.7%p. Overall, Dr.LLM shows that
explicitly supervised routers retrofit frozen LLMs for budget-aware,
accuracy-driven inference without altering base weights.
68ef08b2486b78128f0e340b
Ahmed Heakl
claimed_verified
2025-10-15T03:12:39.051Z
656864e12d73834278a8dea7
https://cdn-avatars.huggingface.co/v1/production/uploads/656864e12d73834278a8dea7/sfAWS2eyPtFHb_2GZIypp.jpeg
Ahmed Heakl
user
ahmedheakl
68ef08b2486b78128f0e340c
Martin Gubri
claimed_verified
2025-10-16T06:41:52.308Z
63ee35c3f599efc7a010c792
https://cdn-avatars.huggingface.co/v1/production/uploads/1676555600618-noauth.jpeg
Martin Gubri
user
mgubri
68ef08b2486b78128f0e340d
Salman Khan
68ef08b2486b78128f0e340e
Sangdoo Yun
68ef08b2486b78128f0e340f
Seong Joon Oh
Dr.LLM: Dynamic Layer Routing in LLMs
Large Language Models (LLMs) process every token through all layers of a
transformer stack, causing wasted computation on simple queries and
insufficient flexibility for harder ones that need deeper reasoning.
Adaptive-depth methods can improve efficiency, but prior approaches rely on
costly inference-time search, architectural changes, or large-scale retraining,
and in practice often degrade accuracy despite efficiency gains. We introduce
Dr.LLM, Dynamic routing of Layers for LLMs, a retrofittable framework that
equips pretrained models with lightweight per-layer routers deciding to skip,
execute, or repeat a block. Routers are trained with explicit supervision:
using Monte Carlo Tree Search (MCTS), we derive high-quality layer
configurations that preserve or improve accuracy under a compute budget. Our
design, windowed pooling for stable routing, focal loss with class balancing,
and bottleneck MLP routers, ensures robustness under class imbalance and long
sequences. On ARC (logic) and DART (math), Dr.LLM improves accuracy by up to
+3.4%p while saving 5 layers per example on average. Routers generalize to
out-of-domain tasks (MMLU, GSM8k, AIME, TruthfulQA, SQuADv2, GPQA, PIQA,
AGIEval) with only 0.85% accuracy drop while retaining efficiency, and
outperform prior routing methods by up to +7.7%p. Overall, Dr.LLM shows that
explicitly supervised routers retrofit frozen LLMs for budget-aware,
accuracy-driven inference without altering base weights.