UltraGen: High-Resolution Video Generation with Hierarchical Attention
Recent advances in video generation have made it possible to produce visually
compelling videos, with wide-ranging applications in content creation,
entertainment, and virtual reality. However, most existing diffusion
transformer based video generation models are limited to low-resolution outputs
(<=720P) due to the quadratic computational complexity of the attention
mechanism with respect to the output width and height. This computational
bottleneck makes native high-resolution video generation (1080P/2K/4K)
impractical for both training and inference. To address this challenge, we
present UltraGen, a novel video generation framework that enables i) efficient
and ii) end-to-end native high-resolution video synthesis. Specifically,
UltraGen features a hierarchical dual-branch attention architecture based on
global-local attention decomposition, which decouples full attention into a
local attention branch for high-fidelity regional content and a global
attention branch for overall semantic consistency. We further propose a
spatially compressed global modeling strategy to efficiently learn global
dependencies, and a hierarchical cross-window local attention mechanism to
reduce computational costs while enhancing information flow across different
local windows. Extensive experiments demonstrate that UltraGen can effectively
scale pre-trained low-resolution video models to 1080P and even 4K resolution
for the first time, outperforming existing state-of-the-art methods and
super-resolution based two-stage pipelines in both qualitative and quantitative
evaluations.
68f838e87669bcaeecce0c79
Teng Hu
68f838e87669bcaeecce0c7a
Jiangning Zhang
68f838e87669bcaeecce0c7b
Zihan Su
68f838e87669bcaeecce0c7c
Ran Yi
UltraGen: High-Resolution Video Generation with Hierarchical Attention
Recent advances in video generation have made it possible to produce visually
compelling videos, with wide-ranging applications in content creation,
entertainment, and virtual reality. However, most existing diffusion
transformer based video generation models are limited to low-resolution outputs
(<=720P) due to the quadratic computational complexity of the attention
mechanism with respect to the output width and height. This computational
bottleneck makes native high-resolution video generation (1080P/2K/4K)
impractical for both training and inference. To address this challenge, we
present UltraGen, a novel video generation framework that enables i) efficient
and ii) end-to-end native high-resolution video synthesis. Specifically,
UltraGen features a hierarchical dual-branch attention architecture based on
global-local attention decomposition, which decouples full attention into a
local attention branch for high-fidelity regional content and a global
attention branch for overall semantic consistency. We further propose a
spatially compressed global modeling strategy to efficiently learn global
dependencies, and a hierarchical cross-window local attention mechanism to
reduce computational costs while enhancing information flow across different
local windows. Extensive experiments demonstrate that UltraGen can effectively
scale pre-trained low-resolution video models to 1080P and even 4K resolution
for the first time, outperforming existing state-of-the-art methods and
super-resolution based two-stage pipelines in both qualitative and quantitative
evaluations.
UltraGen, a novel video generation framework, enables efficient high-resolution video synthesis using a hierarchical dual-branch attention architecture and spatially compressed global modeling.
diffusion transformer video generation low-resolution high-resolution attention mechanism computational complexity global-local attention decomposition local attention global attention spatially compressed global modeling hierarchical cross-window local attention pre-trained models super-resolution two-stage pipelines