LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
Latent Consistency Models (LCMs) have achieved impressive performance in
accelerating text-to-image generative tasks, producing high-quality images with
minimal inference steps. LCMs are distilled from pre-trained latent diffusion
models (LDMs), requiring only ~32 A100 GPU training hours. This report further
extends LCMs' potential in two aspects: First, by applying LoRA distillation to
Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded
LCM's scope to larger models with significantly less memory consumption,
achieving superior image generation quality. Second, we identify the LoRA
parameters obtained through LCM distillation as a universal Stable-Diffusion
acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into
various Stable-Diffusion fine-tuned models or LoRAs without training, thus
representing a universally applicable accelerator for diverse image generation
tasks. Compared with previous numerical PF-ODE solvers such as DDIM,
DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that
possesses strong generalization abilities. Project page:
https://github.com/luosiallen/latent-consistency-model.
654d9b69f714720ae27e332e
Simian Luo
admin_assigned
2023-11-10T08:55:54.016Z
652a4f174d0eef4a2bf85096
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ofYN04WWtD60oeDP45pQN.jpeg
Simian Luo
user
SimianLuo
654d9b69f714720ae27e332f
Yiqin Tan
admin_assigned
2023-11-10T08:56:02.144Z
65237947706c755148c2678b
/avatars/b7ba5dbf6286e8fa8f4f4999a473bd94.svg
Yiqin Tan
user
tyq1024
654d9b69f714720ae27e3330
Suraj Patil
admin_assigned
2023-11-10T09:02:32.127Z
5ec0135ded25d76864d553f1
https://cdn-avatars.huggingface.co/v1/production/uploads/1600014820272-5ec0135ded25d76864d553f1.jpeg
Suraj Patil
user
valhalla
654d9b69f714720ae27e3331
Daniel Gu
admin_assigned
2023-11-10T09:03:10.095Z
64433881f99d6629a71d39ed
/avatars/35e7257e357064e77cfdcae384afe36f.svg
Daniel Gu
user
dg845
654d9b69f714720ae27e3332
Patrick von Platen
admin_assigned
2023-11-10T08:57:29.064Z
5dfcb1aada6d0311fd3d5448
https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg
Patrick von Platen
user
patrickvonplaten
654d9b69f714720ae27e3333
Apolinário Passos
admin_assigned
2023-11-10T09:01:55.660Z
624bebf604abc7ebb01789af
https://cdn-avatars.huggingface.co/v1/production/uploads/1649143001781-624bebf604abc7ebb01789af.jpeg
Apolinário from multimodal AI art
user
multimodalart
654d9b69f714720ae27e3334
Longbo Huang
654d9b69f714720ae27e3335
Jian Li
admin_assigned
2023-11-10T09:00:08.819Z
63b5c13a889aa6707f0e8548
/avatars/a475b141fdc859967a9b63252620be05.svg
Jian Li
user
jianl
654d9b69f714720ae27e3336
Hang Zhao
admin_assigned
2023-11-10T09:00:17.152Z
63bc8467b8c61b8aa49c3e11
/avatars/f75a33b6bba1bb6aecbe018dcef39131.svg
Hang Zhao
user
sgzhaohang
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
Latent Consistency Models (LCMs) have achieved impressive performance in
accelerating text-to-image generative tasks, producing high-quality images with
minimal inference steps. LCMs are distilled from pre-trained latent diffusion
models (LDMs), requiring only ~32 A100 GPU training hours. This report further
extends LCMs' potential in two aspects: First, by applying LoRA distillation to
Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded
LCM's scope to larger models with significantly less memory consumption,
achieving superior image generation quality. Second, we identify the LoRA
parameters obtained through LCM distillation as a universal Stable-Diffusion
acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into
various Stable-Diffusion fine-tuned models or LoRAs without training, thus
representing a universally applicable accelerator for diverse image generation
tasks. Compared with previous numerical PF-ODE solvers such as DDIM,
DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that
possesses strong generalization abilities. Project page:
https://github.com/luosiallen/latent-consistency-model.
LCMs enhance text-to-image generation by distilling from LDMs using LoRA for reduced memory and superior quality, and introduce LCM-LoRA as a plug-in accelerator for various tasks.
Latent Consistency Models LCMs latent diffusion models LDMs LoRA Stable-Diffusion SD-V1.5 SSD-1B SDXL LCM-LoRA DDIM DPM-Solver plug-in neural PF-ODE solver