Large Scale Diffusion Distillation via Score-Regularized Continuous-Time
  Consistency
This work represents the first effort to scale up continuous-time consistency
distillation to general application-level image and video diffusion models.
Although continuous-time consistency model (sCM) is theoretically principled
and empirically powerful for accelerating academic-scale diffusion, its
applicability to large-scale text-to-image and video tasks remains unclear due
to infrastructure challenges in Jacobian-vector product (JVP) computation and
the limitations of standard evaluation benchmarks. We first develop a
parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on
models with over 10 billion parameters and high-dimensional video tasks. Our
investigation reveals fundamental quality limitations of sCM in fine-detail
generation, which we attribute to error accumulation and the "mode-covering"
nature of its forward-divergence objective. To remedy this, we propose the
score-regularized continuous-time consistency model (rCM), which incorporates
score distillation as a long-skip regularizer. This integration complements sCM
with the "mode-seeking" reverse divergence, effectively improving visual
quality while maintaining high generation diversity. Validated on large-scale
models (Cosmos-Predict2, Wan2.1) up to 14B parameters and 5-second videos, rCM
matches or surpasses the state-of-the-art distillation method DMD2 on quality
metrics while offering notable advantages in diversity, all without GAN tuning
or extensive hyperparameter searches. The distilled models generate
high-fidelity samples in only 1sim4 steps, accelerating diffusion sampling
by 15timessim50times. These results position rCM as a practical and
theoretically grounded framework for advancing large-scale diffusion
distillation.
68e866ef95e8e6771df3889a
Kaiwen Zheng
claimed_verified
2025-10-18T04:50:31.750Z
652bf7edc3cba555d5673c6e
/avatars/78f6416c30203b30671f8423f061c657.svg
Kaiwen Zheng
user
worstcoder
68e866ef95e8e6771df3889b
Yuji Wang
68e866ef95e8e6771df3889c
Qianli Ma
68e866ef95e8e6771df3889d
Huayu Chen
claimed_verified
2025-10-10T04:33:05.478Z
65571135bfb62d747abc8129
/avatars/5f4542daa34597f17e6280b9cce18c91.svg
Hugging
user
ChenDRAG
68e866ef95e8e6771df3889e
Jintao Zhang
claimed_verified
2025-10-13T10:10:07.066Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
user
jt-zhang
68e866ef95e8e6771df3889f
Yogesh Balaji
68e866ef95e8e6771df388a0
Jianfei Chen
68e866ef95e8e6771df388a1
Ming-Yu Liu
68e866ef95e8e6771df388a2
Jun Zhu
68e866ef95e8e6771df388a3
Qinsheng Zhang
Large Scale Diffusion Distillation via Score-Regularized Continuous-Time
  Consistency
This work represents the first effort to scale up continuous-time consistency
distillation to general application-level image and video diffusion models.
Although continuous-time consistency model (sCM) is theoretically principled
and empirically powerful for accelerating academic-scale diffusion, its
applicability to large-scale text-to-image and video tasks remains unclear due
to infrastructure challenges in Jacobian-vector product (JVP) computation and
the limitations of standard evaluation benchmarks. We first develop a
parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on
models with over 10 billion parameters and high-dimensional video tasks. Our
investigation reveals fundamental quality limitations of sCM in fine-detail
generation, which we attribute to error accumulation and the "mode-covering"
nature of its forward-divergence objective. To remedy this, we propose the
score-regularized continuous-time consistency model (rCM), which incorporates
score distillation as a long-skip regularizer. This integration complements sCM
with the "mode-seeking" reverse divergence, effectively improving visual
quality while maintaining high generation diversity. Validated on large-scale
models (Cosmos-Predict2, Wan2.1) up to 14B parameters and 5-second videos, rCM
matches or surpasses the state-of-the-art distillation method DMD2 on quality
metrics while offering notable advantages in diversity, all without GAN tuning
or extensive hyperparameter searches. The distilled models generate
high-fidelity samples in only 1sim4 steps, accelerating diffusion sampling
by 15timessim50times. These results position rCM as a practical and
theoretically grounded framework for advancing large-scale diffusion
distillation.
Score-regularized continuous-time consistency model (rCM) improves large-scale diffusion distillation by addressing fine-detail generation and diversity issues, achieving high fidelity and accelerated sampling.
continuous-time consistency model sCM Jacobian-vector product FlashAttention-2 score-regularized continuous-time consistency model rCM score distillation mode-seeking mode-covering diffusion distillation DMD2 diffusion sampling