Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image
  Synthesis
Recently, a series of diffusion-aware distillation algorithms have emerged to
alleviate the computational overhead associated with the multi-step inference
process of Diffusion Models (DMs). Current distillation techniques often
dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii)
ODE Trajectory Reformulation. However, these approaches suffer from severe
performance degradation or domain shifts. To address these limitations, we
propose Hyper-SD, a novel framework that synergistically amalgamates the
advantages of ODE Trajectory Preservation and Reformulation, while maintaining
near-lossless performance during step compression. Firstly, we introduce
Trajectory Segmented Consistency Distillation to progressively perform
consistent distillation within pre-defined time-step segments, which
facilitates the preservation of the original ODE trajectory from a higher-order
perspective. Secondly, we incorporate human feedback learning to boost the
performance of the model in a low-step regime and mitigate the performance loss
incurred by the distillation process. Thirdly, we integrate score distillation
to further improve the low-step generation capability of the model and offer
the first attempt to leverage a unified LoRA to support the inference process
at all steps. Extensive experiments and user studies demonstrate that Hyper-SD
achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5.
For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and
+0.51 in Aes Score in the 1-step inference.
662717e588596f571aea8ec5
Yuxi Ren
admin_assigned
2024-04-23T08:04:10.414Z
6618d5e83b412cdc85334ca8
/avatars/5fe356d58c4c822a60370dbee8d78a69.svg
renyuxi
user
renyuxi
662717e588596f571aea8ec6
Xin Xia
admin_assigned
2024-04-23T08:04:46.842Z
63089a78ff78e2aead8d10e7
/avatars/f326fd08abee7e31599a78923be30003.svg
XinXia
user
XiaXin-Aloys
662717e588596f571aea8ec7
Yanzuo Lu
claimed_verified
2024-04-23T07:13:59.809Z
6614cbd40bbea65e71db4e1f
/avatars/ca8ff74887bbf8eb3f5b04ae9bb6d05b.svg
Yanzuo Lu
user
oliveryanzuolu
662717e588596f571aea8ec8
Jiacheng Zhang
662717e588596f571aea8ec9
Jie Wu
662717e588596f571aea8eca
Pan Xie
admin_assigned
2024-04-23T08:05:25.748Z
62d61f703bf5e059f7d24efd
/avatars/b4a7651de5dfd8a1c26e9ca559886b26.svg
Xie Pan
user
panxie
662717e588596f571aea8ecb
Xing Wang
662717e588596f571aea8ecc
Xuefeng Xiao
admin_assigned
2024-04-23T08:05:44.505Z
646b7f71df2609a541c1ab9f
/avatars/48b82e5fd9b06f41ff825507c36816cd.svg
Xuefeng Xiao
user
xiaoxuefeng
Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image
  Synthesis
Recently, a series of diffusion-aware distillation algorithms have emerged to
alleviate the computational overhead associated with the multi-step inference
process of Diffusion Models (DMs). Current distillation techniques often
dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii)
ODE Trajectory Reformulation. However, these approaches suffer from severe
performance degradation or domain shifts. To address these limitations, we
propose Hyper-SD, a novel framework that synergistically amalgamates the
advantages of ODE Trajectory Preservation and Reformulation, while maintaining
near-lossless performance during step compression. Firstly, we introduce
Trajectory Segmented Consistency Distillation to progressively perform
consistent distillation within pre-defined time-step segments, which
facilitates the preservation of the original ODE trajectory from a higher-order
perspective. Secondly, we incorporate human feedback learning to boost the
performance of the model in a low-step regime and mitigate the performance loss
incurred by the distillation process. Thirdly, we integrate score distillation
to further improve the low-step generation capability of the model and offer
the first attempt to leverage a unified LoRA to support the inference process
at all steps. Extensive experiments and user studies demonstrate that Hyper-SD
achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5.
For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and
+0.51 in Aes Score in the 1-step inference.
Hyper-SD distills Diffusion Models by preserving and reformulating ODE trajectories, incorporating human feedback, and using score distillation, achieving state-of-the-art performance in step compression.
diffusion-aware distillation Diffusion Models ODE Trajectory Preservation ODE Trajectory Reformulation Hyper-SD Trajectory Segmented Consistency Distillation human feedback learning score distillation LoRA CLIP Score Aes Score SDXL SD1.5