Common Diffusion Noise Schedules and Sample Steps are Flawed
We discover that common diffusion noise schedules do not enforce the last
timestep to have zero signal-to-noise ratio (SNR), and some implementations of
diffusion samplers do not start from the last timestep. Such designs are flawed
and do not reflect the fact that the model is given pure Gaussian noise at
inference, creating a discrepancy between training and inference. We show that
the flawed design causes real problems in existing implementations. In Stable
Diffusion, it severely limits the model to only generate images with medium
brightness and prevents it from generating very bright and dark samples. We
propose a few simple fixes: (1) rescale the noise schedule to enforce zero
terminal SNR; (2) train the model with v prediction; (3) change the sampler to
always start from the last timestep; (4) rescale classifier-free guidance to
prevent over-exposure. These simple changes ensure the diffusion process is
congruent between training and inference and allow the model to generate
samples more faithful to the original data distribution.
64645c72884f2e3e1ced059f
Shanchuan Lin
claimed_verified
2023-05-17T08:03:40.137Z
645863f7dc18eb1a9b5d29df
https://cdn-avatars.huggingface.co/v1/production/uploads/645863f7dc18eb1a9b5d29df/t49Nnyl4tbkUn7CmQqKZh.jpeg
Peter Lin
user
PeterL1n
64645c72884f2e3e1ced05a0
Bingchen Liu
claimed_verified
2023-05-17T22:34:38.860Z
62f9e1f90697d2242198c1db
https://cdn-avatars.huggingface.co/v1/production/uploads/1660543471146-noauth.jpeg
bingchen liu
user
codegeasslbc
64645c72884f2e3e1ced05a1
Jiashi Li
64645c72884f2e3e1ced05a2
Xiao Yang
claimed_verified
2023-05-17T08:03:28.428Z
62b3d2490385d6048985b42b
/avatars/1c4e534d086d53ef1057fbc00334350a.svg
Xiao Yang
user
seravee008
Common Diffusion Noise Schedules and Sample Steps are Flawed
We discover that common diffusion noise schedules do not enforce the last
timestep to have zero signal-to-noise ratio (SNR), and some implementations of
diffusion samplers do not start from the last timestep. Such designs are flawed
and do not reflect the fact that the model is given pure Gaussian noise at
inference, creating a discrepancy between training and inference. We show that
the flawed design causes real problems in existing implementations. In Stable
Diffusion, it severely limits the model to only generate images with medium
brightness and prevents it from generating very bright and dark samples. We
propose a few simple fixes: (1) rescale the noise schedule to enforce zero
terminal SNR; (2) train the model with v prediction; (3) change the sampler to
always start from the last timestep; (4) rescale classifier-free guidance to
prevent over-exposure. These simple changes ensure the diffusion process is
congruent between training and inference and allow the model to generate
samples more faithful to the original data distribution.
Proposed solutions address flaws in diffusion noise schedules and samplers to ensure consistency between training and inference, enabling more accurate image generation.
diffusion noise schedules signal-to-noise ratio diffusion samplers Gaussian noise v prediction classifier-free guidance image generation data distribution