LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning
Multi-stage reasoning has emerged as an effective strategy for enhancing the
reasoning capability of small language models by decomposing complex problems
into sequential sub-stages. However, this comes at the cost of increased
latency. We observe that existing adaptive acceleration techniques, such as
layer skipping, struggle to balance efficiency and accuracy in this setting due
to two key challenges: (1) stage-wise variation in skip sensitivity, and (2)
the generation of redundant output tokens. To address these, we propose
LiteStage, a latency-aware layer skipping framework for multi-stage reasoning.
LiteStage combines a stage-wise offline search that allocates optimal layer
budgets with an online confidence-based generation early exit to suppress
unnecessary decoding. Experiments on three benchmarks, e.g., OBQA, CSQA, and
StrategyQA, show that LiteStage achieves up to 1.70x speedup with less than
4.0% accuracy loss, outperforming prior training-free layer skipping methods.
68f1ccce6e0bef323a68ff16
Beomseok Kang
68f1ccce6e0bef323a68ff17
Jiwon Song
claimed_verified
2025-10-17T05:56:29.645Z
662672eaebdfec5cfdf1d034
/avatars/61bc7add693c555e29ad3c1112215684.svg
Jiwon Song
user
jiwonsong
68f1ccce6e0bef323a68ff18
Jae-Joon Kim
LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning
Multi-stage reasoning has emerged as an effective strategy for enhancing the
reasoning capability of small language models by decomposing complex problems
into sequential sub-stages. However, this comes at the cost of increased
latency. We observe that existing adaptive acceleration techniques, such as
layer skipping, struggle to balance efficiency and accuracy in this setting due
to two key challenges: (1) stage-wise variation in skip sensitivity, and (2)
the generation of redundant output tokens. To address these, we propose
LiteStage, a latency-aware layer skipping framework for multi-stage reasoning.
LiteStage combines a stage-wise offline search that allocates optimal layer
budgets with an online confidence-based generation early exit to suppress
unnecessary decoding. Experiments on three benchmarks, e.g., OBQA, CSQA, and
StrategyQA, show that LiteStage achieves up to 1.70x speedup with less than
4.0% accuracy loss, outperforming prior training-free layer skipping methods.
LiteStage, a latency-aware layer skipping framework, enhances multi-stage reasoning by optimizing layer budgets and suppressing redundant output tokens, achieving significant speedup with minimal accuracy loss.
multi-stage reasoning layer skipping stage-wise variation skip sensitivity redundant output tokens confidence-based generation early exit OBQA CSQA StrategyQA