Minute-Long Videos with Dual Parallelisms
Diffusion Transformer (DiT)-based video diffusion models generate
high-quality videos at scale but incur prohibitive processing latency and
memory costs for long videos. To address this, we propose a novel distributed
inference strategy, termed DualParal. The core idea is that, instead of
generating an entire video on a single GPU, we parallelize both temporal frames
and model layers across GPUs. However, a naive implementation of this division
faces a key limitation: since diffusion models require synchronized noise
levels across frames, this implementation leads to the serialization of
original parallelisms. We leverage a block-wise denoising scheme to handle
this. Namely, we process a sequence of frame blocks through the pipeline with
progressively decreasing noise levels. Each GPU handles a specific block and
layer subset while passing previous results to the next GPU, enabling
asynchronous computation and communication. To further optimize performance, we
incorporate two key enhancements. Firstly, a feature cache is implemented on
each GPU to store and reuse features from the prior block as context,
minimizing inter-GPU communication and redundant computation. Secondly, we
employ a coordinated noise initialization strategy, ensuring globally
consistent temporal dynamics by sharing initial noise patterns across GPUs
without extra resource costs. Together, these enable fast, artifact-free, and
infinitely long video generation. Applied to the latest diffusion transformer
video generator, our method efficiently produces 1,025-frame videos with up to
6.54times lower latency and 1.48times lower memory cost on 8timesRTX
4090 GPUs.
683670a816cb1e8ad3235e45
Zeqing Wang
683670a816cb1e8ad3235e46
Bowen Zheng
683670a816cb1e8ad3235e47
Xingyi Yang
claimed_verified
2025-06-05T07:28:50.831Z
634cfebc350bcee9bed20a4d
https://cdn-avatars.huggingface.co/v1/production/uploads/634cfebc350bcee9bed20a4d/fN47nN5rhw-HJaFLBZWQy.png
Xingyi Yang
user
adamdad
683670a816cb1e8ad3235e48
Yuecong Xu
683670a816cb1e8ad3235e49
Xinchao Wang
Minute-Long Videos with Dual Parallelisms
Diffusion Transformer (DiT)-based video diffusion models generate
high-quality videos at scale but incur prohibitive processing latency and
memory costs for long videos. To address this, we propose a novel distributed
inference strategy, termed DualParal. The core idea is that, instead of
generating an entire video on a single GPU, we parallelize both temporal frames
and model layers across GPUs. However, a naive implementation of this division
faces a key limitation: since diffusion models require synchronized noise
levels across frames, this implementation leads to the serialization of
original parallelisms. We leverage a block-wise denoising scheme to handle
this. Namely, we process a sequence of frame blocks through the pipeline with
progressively decreasing noise levels. Each GPU handles a specific block and
layer subset while passing previous results to the next GPU, enabling
asynchronous computation and communication. To further optimize performance, we
incorporate two key enhancements. Firstly, a feature cache is implemented on
each GPU to store and reuse features from the prior block as context,
minimizing inter-GPU communication and redundant computation. Secondly, we
employ a coordinated noise initialization strategy, ensuring globally
consistent temporal dynamics by sharing initial noise patterns across GPUs
without extra resource costs. Together, these enable fast, artifact-free, and
infinitely long video generation. Applied to the latest diffusion transformer
video generator, our method efficiently produces 1,025-frame videos with up to
6.54times lower latency and 1.48times lower memory cost on 8timesRTX
4090 GPUs.
A distributed inference strategy, DualParal, is proposed to address high processing latency and memory costs in diffusion transformer-based video diffusion models by parallelizing frames and layers across GPUs with a block-wise denoising scheme and feature cache.
Diffusion Transformer (DiT) video diffusion models distributed inference DualParal parallelization temporal frames model layers block-wise denoising asynchronous computation feature cache coordinated noise initialization