LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent
  Diffusion Models
Advances in latent diffusion models (LDMs) have revolutionized
high-resolution image generation, but the design space of the autoencoder that
is central to these systems remains underexplored. In this paper, we introduce
LiteVAE, a family of autoencoders for LDMs that leverage the 2D discrete
wavelet transform to enhance scalability and computational efficiency over
standard variational autoencoders (VAEs) with no sacrifice in output quality.
We also investigate the training methodologies and the decoder architecture of
LiteVAE and propose several enhancements that improve the training dynamics and
reconstruction quality. Our base LiteVAE model matches the quality of the
established VAEs in current LDMs with a six-fold reduction in encoder
parameters, leading to faster training and lower GPU memory requirements, while
our larger model outperforms VAEs of comparable complexity across all evaluated
metrics (rFID, LPIPS, PSNR, and SSIM).
665011a5a35b05faab595617
Seyedmorteza Sadat
claimed_verified
2024-05-24T09:31:39.850Z
63b4b02a103617b0a5b0ee2e
https://cdn-avatars.huggingface.co/v1/production/uploads/63b4b02a103617b0a5b0ee2e/LasBC-pCnPJ6XuCt6qqcp.jpeg
Seyedmorteza Sadat
user
msadat97
665011a5a35b05faab595618
Jakob Buhmann
admin_assigned
2024-05-24T11:04:09.083Z
62013227611d68ef490683a2
/avatars/098e8a644692b2a9df052ceea462cbf6.svg
Jakob Buhmann
user
jjbuhmann
665011a5a35b05faab595619
Derek Bradley
665011a5a35b05faab59561a
Otmar Hilliges
665011a5a35b05faab59561b
Romann M. Weber
admin_assigned
2024-05-24T11:04:30.751Z
630f7646197cd3f24e7f8e9f
/avatars/59bbd4ed38277b313051aac78f6808ac.svg
Romann Weber
user
RMW
LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent
  Diffusion Models
Advances in latent diffusion models (LDMs) have revolutionized
high-resolution image generation, but the design space of the autoencoder that
is central to these systems remains underexplored. In this paper, we introduce
LiteVAE, a family of autoencoders for LDMs that leverage the 2D discrete
wavelet transform to enhance scalability and computational efficiency over
standard variational autoencoders (VAEs) with no sacrifice in output quality.
We also investigate the training methodologies and the decoder architecture of
LiteVAE and propose several enhancements that improve the training dynamics and
reconstruction quality. Our base LiteVAE model matches the quality of the
established VAEs in current LDMs with a six-fold reduction in encoder
parameters, leading to faster training and lower GPU memory requirements, while
our larger model outperforms VAEs of comparable complexity across all evaluated
metrics (rFID, LPIPS, PSNR, and SSIM).
LiteVAE, an autoencoder family for LDMs using the 2D discrete wavelet transform, enhances scalability and efficiency without compromising quality, offering improved training dynamics and reconstruction quality.
latent diffusion models LiteVAE 2D discrete wavelet transform variational autoencoders training dynamics rFID LPIPS PSNR SSIM