Long Video Diffusion Generation with Segmented Cross-Attention and
  Content-Rich Video Data Curation
We introduce Presto, a novel video diffusion model designed to generate
15-second videos with long-range coherence and rich content. Extending video
generation methods to maintain scenario diversity over long durations presents
significant challenges. To address this, we propose a Segmented Cross-Attention
(SCA) strategy, which splits hidden states into segments along the temporal
dimension, allowing each segment to cross-attend to a corresponding
sub-caption. SCA requires no additional parameters, enabling seamless
incorporation into current DiT-based architectures. To facilitate high-quality
long video generation, we build the LongTake-HD dataset, consisting of 261k
content-rich videos with scenario coherence, annotated with an overall video
caption and five progressive sub-captions. Experiments show that our Presto
achieves 78.5% on the VBench Semantic Score and 100% on the Dynamic Degree,
outperforming existing state-of-the-art video generation methods. This
demonstrates that our proposed Presto significantly enhances content richness,
maintains long-range coherence, and captures intricate textual details. More
details are displayed on our project page: https://presto-video.github.io/.
674e81bbc5490201063a1c71
Xin Yan
claimed_verified
2024-12-03T11:15:08.535Z
62f1fcbee2c9b4f83b4a6276
https://cdn-avatars.huggingface.co/v1/production/uploads/62f1fcbee2c9b4f83b4a6276/Vb648pHEskN7U0rZouYha.jpeg
Xin Yan
user
Cakeyan
674e81bbc5490201063a1c72
Yuxuan Cai
674e81bbc5490201063a1c73
Qiuyue Wang
674e81bbc5490201063a1c74
Yuan Zhou
674e81bbc5490201063a1c75
Wenhao Huang
extracted_pending
2024-12-03T03:57:51.296Z
639005861c5a6237272557d1
https://cdn-avatars.huggingface.co/v1/production/uploads/1670382915881-noauth.jpeg
Huan Yang
user
hyang0511
674e81bbc5490201063a1c76
Huan Yang
Long Video Diffusion Generation with Segmented Cross-Attention and
  Content-Rich Video Data Curation
We introduce Presto, a novel video diffusion model designed to generate
15-second videos with long-range coherence and rich content. Extending video
generation methods to maintain scenario diversity over long durations presents
significant challenges. To address this, we propose a Segmented Cross-Attention
(SCA) strategy, which splits hidden states into segments along the temporal
dimension, allowing each segment to cross-attend to a corresponding
sub-caption. SCA requires no additional parameters, enabling seamless
incorporation into current DiT-based architectures. To facilitate high-quality
long video generation, we build the LongTake-HD dataset, consisting of 261k
content-rich videos with scenario coherence, annotated with an overall video
caption and five progressive sub-captions. Experiments show that our Presto
achieves 78.5% on the VBench Semantic Score and 100% on the Dynamic Degree,
outperforming existing state-of-the-art video generation methods. This
demonstrates that our proposed Presto significantly enhances content richness,
maintains long-range coherence, and captures intricate textual details. More
details are displayed on our project page: https://presto-video.github.io/.
Presto, a novel video diffusion model with a segmented cross-attention strategy, generates high-quality 15-second videos with content richness and long-range coherence, outperforming existing methods.
video diffusion model Segmented Cross-Attention (SCA) DiT-based architectures LongTake-HD dataset VBench Semantic Score Dynamic Degree