AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset
Diffusion models have achieved remarkable progress in the field of video
generation. However, their iterative denoising nature requires a large number
of inference steps to generate a video, which is slow and computationally
expensive. In this paper, we begin with a detailed analysis of the challenges
present in existing diffusion distillation methods and propose a novel
efficient method, namely AccVideo, to reduce the inference steps for
accelerating video diffusion models with synthetic dataset. We leverage the
pretrained video diffusion model to generate multiple valid denoising
trajectories as our synthetic dataset, which eliminates the use of useless data
points during distillation. Based on the synthetic dataset, we design a
trajectory-based few-step guidance that utilizes key data points from the
denoising trajectories to learn the noise-to-video mapping, enabling video
generation in fewer steps. Furthermore, since the synthetic dataset captures
the data distribution at each diffusion timestep, we introduce an adversarial
training strategy to align the output distribution of the student model with
that of our synthetic dataset, thereby enhancing the video quality. Extensive
experiments demonstrate that our model achieves 8.5x improvements in generation
speed compared to the teacher model while maintaining comparable performance.
Compared to previous accelerating methods, our approach is capable of
generating videos with higher quality and resolution, i.e., 5-seconds,
720x1280, 24fps.
67e3641cd8da46951f860d84
Haiyu Zhang
admin_assigned
2025-03-27T10:02:45.776Z
645b8bf6438d6cfbe1ae47ae
/avatars/3e4dc0523ba8e314b298567cf8f0c0ae.svg
Haiyu Zhang
user
aejion
67e3641cd8da46951f860d85
Xinyuan Chen
admin_assigned
2025-03-27T10:02:58.635Z
643e943a70c6a27621eb1c89
/avatars/73ec521ab5ba84cc7908c52c0acef6ef.svg
Xinyuan Chen
user
AriaChen
67e3641cd8da46951f860d86
Yaohui Wang
admin_assigned
2025-03-27T10:03:18.769Z
63201256c6b20f03c829c4b8
/avatars/a42092119777d65e60b12eb5ba0e45f1.svg
Yaohui Wang
user
YaohuiW
67e3641cd8da46951f860d87
Xihui Liu
admin_assigned
2025-03-27T10:03:26.468Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
67e3641cd8da46951f860d88
Yunhong Wang
67e3641cd8da46951f860d89
Yu Qiao
AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset
Diffusion models have achieved remarkable progress in the field of video
generation. However, their iterative denoising nature requires a large number
of inference steps to generate a video, which is slow and computationally
expensive. In this paper, we begin with a detailed analysis of the challenges
present in existing diffusion distillation methods and propose a novel
efficient method, namely AccVideo, to reduce the inference steps for
accelerating video diffusion models with synthetic dataset. We leverage the
pretrained video diffusion model to generate multiple valid denoising
trajectories as our synthetic dataset, which eliminates the use of useless data
points during distillation. Based on the synthetic dataset, we design a
trajectory-based few-step guidance that utilizes key data points from the
denoising trajectories to learn the noise-to-video mapping, enabling video
generation in fewer steps. Furthermore, since the synthetic dataset captures
the data distribution at each diffusion timestep, we introduce an adversarial
training strategy to align the output distribution of the student model with
that of our synthetic dataset, thereby enhancing the video quality. Extensive
experiments demonstrate that our model achieves 8.5x improvements in generation
speed compared to the teacher model while maintaining comparable performance.
Compared to previous accelerating methods, our approach is capable of
generating videos with higher quality and resolution, i.e., 5-seconds,
720x1280, 24fps.
AccVideo accelerates video diffusion models by using a synthetic dataset and trajectory-based guidance, significantly reducing inference steps while maintaining or improving video quality and resolution.
diffusion models denoising inference steps diffusion distillation synthetic dataset pretrained video diffusion model denoising trajectories noise-to-video mapping adversarial training output distribution