Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion
  Distillation
Diffusion models are the main driver of progress in image and video
synthesis, but suffer from slow inference speed. Distillation methods, like the
recently introduced adversarial diffusion distillation (ADD) aim to shift the
model from many-shot to single-step inference, albeit at the cost of expensive
and difficult optimization due to its reliance on a fixed pretrained DINOv2
discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a
novel distillation approach overcoming the limitations of ADD. In contrast to
pixel-based ADD, LADD utilizes generative features from pretrained latent
diffusion models. This approach simplifies training and enhances performance,
enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to
Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the
performance of state-of-the-art text-to-image generators using only four
unguided sampling steps. Moreover, we systematically investigate its scaling
behavior and demonstrate LADD's effectiveness in various applications such as
image editing and inpainting.
65f90374d09e592f404b84b2
Axel Sauer
admin_assigned
2024-03-19T09:05:03.614Z
61fc209cef99814f1705e934
https://cdn-avatars.huggingface.co/v1/production/uploads/61fc209cef99814f1705e934/PSgCbKtcCb5dg84513fo-.jpeg
Axel Sauer
user
xl-sr
65f90374d09e592f404b84b3
Frederic Boesel
admin_assigned
2024-03-19T09:05:10.453Z
653beb48ccfb7ba1cbbc9218
/avatars/f14a52db958f56d3082c2545b88ea99a.svg
Frederic Boesel
user
fboeEl
65f90374d09e592f404b84b4
Tim Dockhorn
admin_assigned
2024-03-19T09:05:17.176Z
633f7a8f4be90e06da248e0f
https://cdn-avatars.huggingface.co/v1/production/uploads/633f7a8f4be90e06da248e0f/Kf-vMGCSIar8UGezk-TK9.jpeg
Tim Dockhorn
user
timudk
65f90374d09e592f404b84b5
Andreas Blattmann
admin_assigned
2024-03-19T09:05:23.010Z
635fbc957c05eb9f5997b568
/avatars/0bf85960fa7ad445b0abb2d70c9d5721.svg
Andreas Blattmann
user
ablattmann
65f90374d09e592f404b84b6
Patrick Esser
admin_assigned
2024-03-19T09:05:29.105Z
62f2576394fe062ca9adf3d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1660049160797-noauth.png
Patrick Esser
user
pesser
65f90374d09e592f404b84b7
Robin Rombach
admin_assigned
2024-03-19T09:05:35.009Z
62cfefa74b3e8dc1e32e38bf
https://cdn-avatars.huggingface.co/v1/production/uploads/1657794439283-noauth.png
Robin Rombach
user
rromb
Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion
  Distillation
Diffusion models are the main driver of progress in image and video
synthesis, but suffer from slow inference speed. Distillation methods, like the
recently introduced adversarial diffusion distillation (ADD) aim to shift the
model from many-shot to single-step inference, albeit at the cost of expensive
and difficult optimization due to its reliance on a fixed pretrained DINOv2
discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a
novel distillation approach overcoming the limitations of ADD. In contrast to
pixel-based ADD, LADD utilizes generative features from pretrained latent
diffusion models. This approach simplifies training and enhances performance,
enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to
Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the
performance of state-of-the-art text-to-image generators using only four
unguided sampling steps. Moreover, we systematically investigate its scaling
behavior and demonstrate LADD's effectiveness in various applications such as
image editing and inpainting.
Latent Adversarial Diffusion Distillation (LADD) simplifies the training and improves the performance of diffusion models by using generative features from latent diffusion models, enabling fast and high-resolution image synthesis.
diffusion models adversarial diffusion distillation (ADD) pretrained DINOv2 discriminator Latent Adversarial Diffusion Distillation (LADD) pixel-based ADD latent diffusion models high-resolution multi-aspect ratio image synthesis Stable Diffusion 3 (8B) SD3-Turbo image editing image inpainting