MLCM: Multistep Consistency Distillation of Latent Diffusion Model
Distilling large latent diffusion models (LDMs) into ones that are fast to
sample from is attracting growing research interest. However, the majority of
existing methods face a dilemma where they either (i) depend on multiple
individual distilled models for different sampling budgets, or (ii) sacrifice
generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8)
sampling steps. To address these, we extend the recent multistep consistency
distillation (MCD) strategy to representative LDMs, establishing the Multistep
Latent Consistency Models (MLCMs) approach for low-cost high-quality image
synthesis. MLCM serves as a unified model for various sampling steps due to the
promise of MCD. We further augment MCD with a progressive training strategy to
strengthen inter-segment consistency to boost the quality of few-step
generations. We take the states from the sampling trajectories of the teacher
model as training data for MLCMs to lift the requirements for high-quality
training datasets and to bridge the gap between the training and inference of
the distilled model. MLCM is compatible with preference learning strategies for
further improvement of visual quality and aesthetic appeal. Empirically, MLCM
can generate high-quality, delightful images with only 2-8 sampling steps. On
the MSCOCO-2017 5K benchmark, MLCM distilled from SDXL gets a CLIP Score of
33.30, Aesthetic Score of 6.19, and Image Reward of 1.20 with only 4 steps,
substantially surpassing 4-step LCM [23], 8-step SDXL-Lightning [17], and
8-step HyperSD [33]. We also demonstrate the versatility of MLCMs in
applications including controllable generation, image style transfer, and
Chinese-to-image generation.
6667c2e82cef2f5b6d8291a4
Qingsong Xie
admin_assigned
2024-06-11T10:43:15.366Z
64184911db24526c7c9ca035
/avatars/2be15485bdd9e00857d4bff107d1577e.svg
qingsong xie
user
kongming11
6667c2e82cef2f5b6d8291a5
Zhenyi Liao
6667c2e82cef2f5b6d8291a6
Zhijie Deng
admin_assigned
2024-06-11T10:43:28.705Z
64bba541da140e461924dfed
/avatars/367993765b0ca3734b2b100db33ed787.svg
zhijie deng
user
zhijie3
6667c2e82cef2f5b6d8291a7
Chen chen
claimed_verified
2024-06-11T13:26:42.883Z
649d47e06503630c5269d93a
/avatars/6812c9f79f2bd753ed68afb03baac202.svg
Chen Chen
user
beckhamchen
6667c2e82cef2f5b6d8291a8
Shixiang Tang
admin_assigned
2024-06-11T10:43:59.396Z
6436403bf3b08e267d9f0329
/avatars/a5d9c3d47073e71e4cea124d9c17356d.svg
SHIXIANG TANG
user
tangshixiang
6667c2e82cef2f5b6d8291a9
Haonan Lu
admin_assigned
2024-06-11T10:44:05.156Z
603864181b642aef9350af81
/avatars/27ec0b885cbedef007876c8c98c69d15.svg
Haonan Lu
user
luhaonan20
MLCM: Multistep Consistency Distillation of Latent Diffusion Model
Distilling large latent diffusion models (LDMs) into ones that are fast to
sample from is attracting growing research interest. However, the majority of
existing methods face a dilemma where they either (i) depend on multiple
individual distilled models for different sampling budgets, or (ii) sacrifice
generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8)
sampling steps. To address these, we extend the recent multistep consistency
distillation (MCD) strategy to representative LDMs, establishing the Multistep
Latent Consistency Models (MLCMs) approach for low-cost high-quality image
synthesis. MLCM serves as a unified model for various sampling steps due to the
promise of MCD. We further augment MCD with a progressive training strategy to
strengthen inter-segment consistency to boost the quality of few-step
generations. We take the states from the sampling trajectories of the teacher
model as training data for MLCMs to lift the requirements for high-quality
training datasets and to bridge the gap between the training and inference of
the distilled model. MLCM is compatible with preference learning strategies for
further improvement of visual quality and aesthetic appeal. Empirically, MLCM
can generate high-quality, delightful images with only 2-8 sampling steps. On
the MSCOCO-2017 5K benchmark, MLCM distilled from SDXL gets a CLIP Score of
33.30, Aesthetic Score of 6.19, and Image Reward of 1.20 with only 4 steps,
substantially surpassing 4-step LCM [23], 8-step SDXL-Lightning [17], and
8-step HyperSD [33]. We also demonstrate the versatility of MLCMs in
applications including controllable generation, image style transfer, and
Chinese-to-image generation.
A new approach, Multistep Latent Consistency Models (MLCMs), extends the multistep consistency distillation strategy to latent diffusion models, enabling high-quality image synthesis with fewer sampling steps.
multistep consistency distillation Multistep Latent Consistency Models progressive training sampling trajectories preference learning CLIP Score Aesthetic Score Image Reward controllable generation image style transfer Chinese-to-image generation