OSV: One Step is Enough for High-Quality Image to Video Generation
Video diffusion models have shown great potential in generating high-quality
videos, making them an increasingly popular focus. However, their inherent
iterative nature leads to substantial computational and time costs. While
efforts have been made to accelerate video diffusion by reducing inference
steps (through techniques like consistency distillation) and GAN training
(these approaches often fall short in either performance or training
stability). In this work, we introduce a two-stage training framework that
effectively combines consistency distillation with GAN training to address
these challenges. Additionally, we propose a novel video discriminator design,
which eliminates the need for decoding the video latents and improves the final
performance. Our model is capable of producing high-quality videos in merely
one-step, with the flexibility to perform multi-step refinement for further
performance enhancement. Our quantitative evaluation on the OpenWebVid-1M
benchmark shows that our model significantly outperforms existing methods.
Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of
the consistency distillation based method, AnimateLCM (FVD 184.79), and
approaches the 25-step performance of advanced Stable Video Diffusion (FVD
156.94).
66ea33e408671611fc90a2c6
Xiaofeng Mao
66ea33e408671611fc90a2c7
Zhengkai Jiang
claimed_verified
2024-10-01T13:10:43.706Z
6371dda2ffc0489ed7d712c2
/avatars/dcb80cae00535c22af99baf86cb70d93.svg
zkjiang
user
justin-zk
66ea33e408671611fc90a2c8
Fu-Yun Wang
claimed_verified
2024-09-18T07:01:39.124Z
63e9e92f20c109718713f5eb
/avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
Fu-Yun Wang
user
wangfuyun
66ea33e408671611fc90a2c9
Wenbing Zhu
66ea33e408671611fc90a2ca
Jiangning Zhang
66ea33e408671611fc90a2cb
Hao Chen
66ea33e408671611fc90a2cc
Mingmin Chi
66ea33e408671611fc90a2cd
Yabiao Wang
OSV: One Step is Enough for High-Quality Image to Video Generation
Video diffusion models have shown great potential in generating high-quality
videos, making them an increasingly popular focus. However, their inherent
iterative nature leads to substantial computational and time costs. While
efforts have been made to accelerate video diffusion by reducing inference
steps (through techniques like consistency distillation) and GAN training
(these approaches often fall short in either performance or training
stability). In this work, we introduce a two-stage training framework that
effectively combines consistency distillation with GAN training to address
these challenges. Additionally, we propose a novel video discriminator design,
which eliminates the need for decoding the video latents and improves the final
performance. Our model is capable of producing high-quality videos in merely
one-step, with the flexibility to perform multi-step refinement for further
performance enhancement. Our quantitative evaluation on the OpenWebVid-1M
benchmark shows that our model significantly outperforms existing methods.
Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of
the consistency distillation based method, AnimateLCM (FVD 184.79), and
approaches the 25-step performance of advanced Stable Video Diffusion (FVD
156.94).
A two-stage training framework combining consistency distillation and GAN training improves video diffusion model efficiency and performance.
video diffusion models consistency distillation GAN training video discriminator video latents OpenWebVid-1M FVD