Aura Attention: $\mathcal O(n \log n)$ Sparse Attention for Long Video Generation
XINGYANG LI, Muyang Li, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Lvmin Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, Song Han
Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training and inference on long videos prohibitively expensive. In this paper, we identify a phenomenon we call \textit{Spatiotemporal Energy Decay} in video diffusion models: post-softmax attention scores decrease as spatial and temporal distance between tokens increase, akin to the physical decay of signal or waves over space and time in nature. Motivated by this, we propose \textit{\method}, a scalable sparse attention mechanism with $\mathcal{O}(n \log n)$ complexity that translates energy decay into exponentially decaying compute density. \method employs a simple, static attention mask where each token attends to spatially nearby tokens, with the attention window size shrinking with temporal distance. Moreover, \method allows pre-trained video diffusion models to extend their generation length with minimal fine-tuning with LoRA. Extensive experiments show that \method maintains video quality across Wan2.1-14B, HunyuanVideo, achieving up to a 1.9× speedup over full attention. With minimal fine-tuning, it enables video generation up to 4× longer while reducing training costs by up to 4.4× compared to direct fine-tuning and accelerating inference by 3.7× compared to full attention inference. Code and models will be released upon publication.