Stable Consistency Tuning: Understanding and Improving Consistency
  Models
Diffusion models achieve superior generation quality but suffer from slow
generation speed due to the iterative nature of denoising. In contrast,
consistency models, a new generative family, achieve competitive performance
with significantly faster sampling. These models are trained either through
consistency distillation, which leverages pretrained diffusion models, or
consistency training/tuning directly from raw data. In this work, we propose a
novel framework for understanding consistency models by modeling the denoising
process of the diffusion model as a Markov Decision Process (MDP) and framing
consistency model training as the value estimation through Temporal
Difference~(TD) Learning. More importantly, this framework allows us to analyze
the limitations of current consistency training/tuning strategies. Built upon
Easy Consistency Tuning (ECT), we propose Stable Consistency Tuning (SCT),
which incorporates variance-reduced learning using the score identity. SCT
leads to significant performance improvements on benchmarks such as CIFAR-10
and ImageNet-64. On ImageNet-64, SCT achieves 1-step FID 2.42 and 2-step FID
1.55, a new SoTA for consistency models.
671af6c26e2ed37782064466
Fu-Yun Wang
claimed_verified
2024-10-25T09:30:18.737Z
63e9e92f20c109718713f5eb
/avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
Fu-Yun Wang
user
wangfuyun
671af6c26e2ed37782064467
Zhengyang Geng
extracted_pending
2024-10-25T01:39:16.932Z
626039656089b62d9f332854
https://cdn-avatars.huggingface.co/v1/production/uploads/1650473291256-noauth.jpeg
Zhengyang Geng
user
Gsunshine
671af6c26e2ed37782064468
Hongsheng Li
Stable Consistency Tuning: Understanding and Improving Consistency
  Models
Diffusion models achieve superior generation quality but suffer from slow
generation speed due to the iterative nature of denoising. In contrast,
consistency models, a new generative family, achieve competitive performance
with significantly faster sampling. These models are trained either through
consistency distillation, which leverages pretrained diffusion models, or
consistency training/tuning directly from raw data. In this work, we propose a
novel framework for understanding consistency models by modeling the denoising
process of the diffusion model as a Markov Decision Process (MDP) and framing
consistency model training as the value estimation through Temporal
Difference~(TD) Learning. More importantly, this framework allows us to analyze
the limitations of current consistency training/tuning strategies. Built upon
Easy Consistency Tuning (ECT), we propose Stable Consistency Tuning (SCT),
which incorporates variance-reduced learning using the score identity. SCT
leads to significant performance improvements on benchmarks such as CIFAR-10
and ImageNet-64. On ImageNet-64, SCT achieves 1-step FID 2.42 and 2-step FID
1.55, a new SoTA for consistency models.
A novel framework understanding consistency models through MDP and TD Learning enhances performance with Stable Consistency Tuning, achieving state-of-the-art results.
diffusion models consistency models consistency distillation consistency training Markov Decision Process Temporal Difference Learning Easy Consistency Tuning Stable Consistency Tuning variance-reduced learning score identity CIFAR-10 ImageNet-64 FID