Less Is More: Training-Free Sparse Attention with Global Locality for
  Efficient Reasoning
Large reasoning models achieve strong performance through test-time scaling
but incur substantial computational overhead, particularly from excessive token
generation when processing short input prompts. While sparse attention
mechanisms can reduce latency and memory usage, existing approaches suffer from
significant accuracy degradation due to accumulated errors during
long-generation reasoning. These methods generally require either high token
retention rates or expensive retraining. We introduce LessIsMore, a
training-free sparse attention mechanism for reasoning tasks, which leverages
global attention patterns rather than relying on traditional head-specific
local optimizations. LessIsMore aggregates token selections from local
attention heads with recent contextual information, enabling unified cross-head
token ranking for future decoding layers. This unified selection improves
generalization and efficiency by avoiding the need to maintain separate token
subsets per head. Evaluation across diverse reasoning tasks and benchmarks
shows that LessIsMore preserves -- and in some cases improves -- accuracy while
achieving a 1.1times average decoding speed-up compared to full attention.
Moreover, LessIsMore attends to 2times fewer tokens without accuracy loss,
achieving a 1.13times end-to-end speed-up compared to existing sparse
attention methods.
689ab5bffab6fdd2e52ac492
Lijie Yang
claimed_verified
2025-08-13T07:20:09.393Z
65d8523167819fdde61c9e9d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/7CrWbLndFzd3YZDke1OgK.jpeg
Lijie Yang
user
drkylj
689ab5bffab6fdd2e52ac493
Zhihao Zhang
689ab5bffab6fdd2e52ac494
Arti Jain
689ab5bffab6fdd2e52ac495
Shijie Cao
689ab5bffab6fdd2e52ac496
Baihong Yuan
689ab5bffab6fdd2e52ac497
Yiwei Chen
689ab5bffab6fdd2e52ac498
Zhihao Jia
689ab5bffab6fdd2e52ac499
Ravi Netravali
Less Is More: Training-Free Sparse Attention with Global Locality for
  Efficient Reasoning
Large reasoning models achieve strong performance through test-time scaling
but incur substantial computational overhead, particularly from excessive token
generation when processing short input prompts. While sparse attention
mechanisms can reduce latency and memory usage, existing approaches suffer from
significant accuracy degradation due to accumulated errors during
long-generation reasoning. These methods generally require either high token
retention rates or expensive retraining. We introduce LessIsMore, a
training-free sparse attention mechanism for reasoning tasks, which leverages
global attention patterns rather than relying on traditional head-specific
local optimizations. LessIsMore aggregates token selections from local
attention heads with recent contextual information, enabling unified cross-head
token ranking for future decoding layers. This unified selection improves
generalization and efficiency by avoiding the need to maintain separate token
subsets per head. Evaluation across diverse reasoning tasks and benchmarks
shows that LessIsMore preserves -- and in some cases improves -- accuracy while
achieving a 1.1times average decoding speed-up compared to full attention.
Moreover, LessIsMore attends to 2times fewer tokens without accuracy loss,
achieving a 1.13times end-to-end speed-up compared to existing sparse
attention methods.
LessIsMore is a training-free sparse attention mechanism that improves efficiency and generalization in reasoning tasks by aggregating token selections from local attention heads.
sparse attention mechanisms global attention patterns local attention heads token ranking decoding speed-up end-to-end speed-up