DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video
  Generation
Diffusion Models have achieved remarkable results in video synthesis but
require iterative denoising steps, leading to substantial computational
overhead. Consistency Models have made significant progress in accelerating
diffusion models. However, directly applying them to video diffusion models
often results in severe degradation of temporal consistency and appearance
details. In this paper, by analyzing the training dynamics of Consistency
Models, we identify a key conflicting learning dynamics during the distillation
process: there is a significant discrepancy in the optimization gradients and
loss contributions across different timesteps. This discrepancy prevents the
distilled student model from achieving an optimal state, leading to compromised
temporal consistency and degraded appearance details. To address this issue, we
propose a parameter-efficient Dual-Expert Consistency Model~(DCM),
where a semantic expert focuses on learning semantic layout and motion, while a
detail expert specializes in fine detail refinement. Furthermore, we introduce
Temporal Coherence Loss to improve motion consistency for the semantic expert
and apply GAN and Feature Matching Loss to enhance the synthesis quality of the
detail expert.Our approach achieves state-of-the-art visual quality with
significantly reduced sampling steps, demonstrating the effectiveness of expert
specialization in video diffusion model distillation. Our code and models are
available at
https://github.com/Vchitect/DCM{https://github.com/Vchitect/DCM}.
684050b04e6c9ec4722c2154
Zhengyao Lv
684050b04e6c9ec4722c2155
Chenyang Si
claimed_verified
2025-06-10T09:22:42.591Z
635f8ed47c05eb9f59963d3a
https://cdn-avatars.huggingface.co/v1/production/uploads/635f8ed47c05eb9f59963d3a/uQf4p9N9pSaFy87Wg9v4k.jpeg
ChenyangSi
user
ChenyangSi
684050b04e6c9ec4722c2156
Tianlin Pan
684050b04e6c9ec4722c2157
Zhaoxi Chen
684050b04e6c9ec4722c2158
Kwan-Yee K. Wong
684050b04e6c9ec4722c2159
Yu Qiao
684050b04e6c9ec4722c215a
Ziwei Liu
DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video
  Generation
Diffusion Models have achieved remarkable results in video synthesis but
require iterative denoising steps, leading to substantial computational
overhead. Consistency Models have made significant progress in accelerating
diffusion models. However, directly applying them to video diffusion models
often results in severe degradation of temporal consistency and appearance
details. In this paper, by analyzing the training dynamics of Consistency
Models, we identify a key conflicting learning dynamics during the distillation
process: there is a significant discrepancy in the optimization gradients and
loss contributions across different timesteps. This discrepancy prevents the
distilled student model from achieving an optimal state, leading to compromised
temporal consistency and degraded appearance details. To address this issue, we
propose a parameter-efficient Dual-Expert Consistency Model~(DCM),
where a semantic expert focuses on learning semantic layout and motion, while a
detail expert specializes in fine detail refinement. Furthermore, we introduce
Temporal Coherence Loss to improve motion consistency for the semantic expert
and apply GAN and Feature Matching Loss to enhance the synthesis quality of the
detail expert.Our approach achieves state-of-the-art visual quality with
significantly reduced sampling steps, demonstrating the effectiveness of expert
specialization in video diffusion model distillation. Our code and models are
available at
https://github.com/Vchitect/DCM{https://github.com/Vchitect/DCM}.
The Dual-Expert Consistency Model improves video diffusion model distillation by addressing learning dynamics issues, achieving state-of-the-art quality with reduced sampling steps.
Diffusion Models Consistency Models training dynamics dual-expert semantic expert detail expert Temporal Coherence Loss GAN Feature Matching Loss