Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
Consistency models (CMs) are a powerful class of diffusion-based generative
models optimized for fast sampling. Most existing CMs are trained using
discretized timesteps, which introduce additional hyperparameters and are prone
to discretization errors. While continuous-time formulations can mitigate these
issues, their success has been limited by training instability. To address
this, we propose a simplified theoretical framework that unifies previous
parameterizations of diffusion models and CMs, identifying the root causes of
instability. Based on this analysis, we introduce key improvements in diffusion
process parameterization, network architecture, and training objectives. These
changes enable us to train continuous-time CMs at an unprecedented scale,
reaching 1.5B parameters on ImageNet 512x512. Our proposed training algorithm,
using only two sampling steps, achieves FID scores of 2.06 on CIFAR-10, 1.48 on
ImageNet 64x64, and 1.88 on ImageNet 512x512, narrowing the gap in FID scores
with the best existing diffusion models to within 10%.
670ff433d126c976117487b3
Cheng Lu
670ff433d126c976117487b4
Yang Song
Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
Consistency models (CMs) are a powerful class of diffusion-based generative
models optimized for fast sampling. Most existing CMs are trained using
discretized timesteps, which introduce additional hyperparameters and are prone
to discretization errors. While continuous-time formulations can mitigate these
issues, their success has been limited by training instability. To address
this, we propose a simplified theoretical framework that unifies previous
parameterizations of diffusion models and CMs, identifying the root causes of
instability. Based on this analysis, we introduce key improvements in diffusion
process parameterization, network architecture, and training objectives. These
changes enable us to train continuous-time CMs at an unprecedented scale,
reaching 1.5B parameters on ImageNet 512x512. Our proposed training algorithm,
using only two sampling steps, achieves FID scores of 2.06 on CIFAR-10, 1.48 on
ImageNet 64x64, and 1.88 on ImageNet 512x512, narrowing the gap in FID scores
with the best existing diffusion models to within 10%.
consistency models diffusion-based generative models fast sampling discretized timesteps hyperparameters discretization errors continuous-time formulations training instability theoretical diffusion model parameterization network architecture training objectives continuous-time CMs FID scores ImageNet CIFAR-10 sampling steps parameter frequencies search in the diffusion