One-Way Ticket:Time-Independent Unified Encoder for Distilling
  Text-to-Image Diffusion Models
Text-to-Image (T2I) diffusion models have made remarkable advancements in
generative modeling; however, they face a trade-off between inference speed and
image quality, posing challenges for efficient deployment. Existing distilled
T2I models can generate high-fidelity images with fewer sampling steps, but
often struggle with diversity and quality, especially in one-step models. From
our analysis, we observe redundant computations in the UNet encoders. Our
findings suggest that, for T2I diffusion models, decoders are more adept at
capturing richer and more explicit semantic information, while encoders can be
effectively shared across decoders from diverse time steps. Based on these
observations, we introduce the first Time-independent Unified Encoder TiUE for
the student model UNet architecture, which is a loop-free image generation
approach for distilling T2I diffusion models. Using a one-pass scheme, TiUE
shares encoder features across multiple decoder time steps, enabling parallel
sampling and significantly reducing inference time complexity. In addition, we
incorporate a KL divergence term to regularize noise prediction, which enhances
the perceptual realism and diversity of the generated images. Experimental
results demonstrate that TiUE outperforms state-of-the-art methods, including
LCM, SD-Turbo, and SwiftBrushv2, producing more diverse and realistic results
while maintaining the computational efficiency.
6837dd74ec10479b9605da15
Senmao Li
claimed_verified
2025-05-29T07:41:51.659Z
637e1cf4f09bf2498c543a73
/avatars/3c0e4e15602a384839bee0c23029f58a.svg
Senmao Li
user
senmaonk
6837dd74ec10479b9605da16
Lei Wang
6837dd74ec10479b9605da17
Kai Wang
6837dd74ec10479b9605da18
Tao Liu
6837dd74ec10479b9605da19
Jiehang Xie
6837dd74ec10479b9605da1a
Joost van de Weijer
6837dd74ec10479b9605da1b
Fahad Shahbaz Khan
6837dd74ec10479b9605da1c
Shiqi Yang
6837dd74ec10479b9605da1d
Yaxing Wang
6837dd74ec10479b9605da1e
Jian Yang
One-Way Ticket:Time-Independent Unified Encoder for Distilling
  Text-to-Image Diffusion Models
Text-to-Image (T2I) diffusion models have made remarkable advancements in
generative modeling; however, they face a trade-off between inference speed and
image quality, posing challenges for efficient deployment. Existing distilled
T2I models can generate high-fidelity images with fewer sampling steps, but
often struggle with diversity and quality, especially in one-step models. From
our analysis, we observe redundant computations in the UNet encoders. Our
findings suggest that, for T2I diffusion models, decoders are more adept at
capturing richer and more explicit semantic information, while encoders can be
effectively shared across decoders from diverse time steps. Based on these
observations, we introduce the first Time-independent Unified Encoder TiUE for
the student model UNet architecture, which is a loop-free image generation
approach for distilling T2I diffusion models. Using a one-pass scheme, TiUE
shares encoder features across multiple decoder time steps, enabling parallel
sampling and significantly reducing inference time complexity. In addition, we
incorporate a KL divergence term to regularize noise prediction, which enhances
the perceptual realism and diversity of the generated images. Experimental
results demonstrate that TiUE outperforms state-of-the-art methods, including
LCM, SD-Turbo, and SwiftBrushv2, producing more diverse and realistic results
while maintaining the computational efficiency.
Time-independent Unified Encoder TiUE reduces inference time and improves diversity and quality in Text-to-Image diffusion models by sharing encoder features across decoder time steps.
Text-to-Image diffusion models inference speed image quality distilled T2I models UNet encoders decoders semantic information Time-independent Unified Encoder TiUE loop-free image generation one-pass scheme parallel sampling KL divergence perceptual realism LCM SD-Turbo SwiftBrushv2