Guiding a Diffusion Model with a Bad Version of Itself
The primary axes of interest in image-generating diffusion models are image
quality, the amount of variation in the results, and how well the results align
with a given condition, e.g., a class label or a text prompt. The popular
classifier-free guidance approach uses an unconditional model to guide a
conditional model, leading to simultaneously better prompt alignment and
higher-quality images at the cost of reduced variation. These effects seem
inherently entangled, and thus hard to control. We make the surprising
observation that it is possible to obtain disentangled control over image
quality without compromising the amount of variation by guiding generation
using a smaller, less-trained version of the model itself rather than an
unconditional model. This leads to significant improvements in ImageNet
generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using
publicly available networks. Furthermore, the method is also applicable to
unconditional diffusion models, drastically improving their quality.
665fe7147c3f9c721120365d
Tero Karras
admin_assigned
2024-06-05T09:34:08.340Z
66587947b7d313010028ac02
/avatars/cb95f524dd1c80aa6ec8af977c5f2606.svg
Tero Karras
user
tkarras
665fe7147c3f9c721120365e
Miika Aittala
admin_assigned
2024-06-05T09:34:14.745Z
630f80e2a119d49bc1e3836f
/avatars/e0ffe1c19df64606cb30801ad20a9f43.svg
Miika Aittala
user
miika
665fe7147c3f9c721120365f
Tuomas Kynkäänniemi
admin_assigned
2024-06-05T09:34:21.022Z
632c060d930d186432d4e1db
/avatars/da8898374dc3677892e2538c2cfa6984.svg
Tuomas Kynkäänniemi
user
kynkaat
665fe7147c3f9c7211203660
Jaakko Lehtinen
665fe7147c3f9c7211203661
Timo Aila
665fe7147c3f9c7211203662
Samuli Laine
claimed_verified
2025-02-13T08:27:15.858Z
67ac68cda78e0ee095322704
/avatars/7cc9fbd7ba8c02e48b01e56bbccea01e.svg
Samuli Laine
user
slaine2
Guiding a Diffusion Model with a Bad Version of Itself
The primary axes of interest in image-generating diffusion models are image
quality, the amount of variation in the results, and how well the results align
with a given condition, e.g., a class label or a text prompt. The popular
classifier-free guidance approach uses an unconditional model to guide a
conditional model, leading to simultaneously better prompt alignment and
higher-quality images at the cost of reduced variation. These effects seem
inherently entangled, and thus hard to control. We make the surprising
observation that it is possible to obtain disentangled control over image
quality without compromising the amount of variation by guiding generation
using a smaller, less-trained version of the model itself rather than an
unconditional model. This leads to significant improvements in ImageNet
generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using
publicly available networks. Furthermore, the method is also applicable to
unconditional diffusion models, drastically improving their quality.
Using smaller, less-trained versions of the model to guide generation leads to better prompt alignment and higher image quality without reducing variation in diffusion models.
diffusion models image quality variation conditional model unconditional model classifier-free guidance ImageNet FIDs