Alleviating Distortion in Image Generation via Multi-Resolution
  Diffusion Models
This paper presents innovative enhancements to diffusion models by
integrating a novel multi-resolution network and time-dependent layer
normalization. Diffusion models have gained prominence for their effectiveness
in high-fidelity image generation. While conventional approaches rely on
convolutional U-Net architectures, recent Transformer-based designs have
demonstrated superior performance and scalability. However, Transformer
architectures, which tokenize input data (via "patchification"), face a
trade-off between visual fidelity and computational complexity due to the
quadratic nature of self-attention operations concerning token length. While
larger patch sizes enable attention computation efficiency, they struggle to
capture fine-grained visual details, leading to image distortions. To address
this challenge, we propose augmenting the Diffusion model with the
Multi-Resolution network (DiMR), a framework that refines features across
multiple resolutions, progressively enhancing detail from low to high
resolution. Additionally, we introduce Time-Dependent Layer Normalization
(TD-LN), a parameter-efficient approach that incorporates time-dependent
parameters into layer normalization to inject time information and achieve
superior performance. Our method's efficacy is demonstrated on the
class-conditional ImageNet generation benchmark, where DiMR-XL variants
outperform prior diffusion models, setting new state-of-the-art FID scores of
1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512. Project page:
https://qihao067.github.io/projects/DiMR
666bb80de59c42693adae02a
Qihao Liu
claimed_verified
2024-06-14T07:22:24.541Z
639f1e519f1f2baab2f00d22
https://cdn-avatars.huggingface.co/v1/production/uploads/639f1e519f1f2baab2f00d22/pFjd51WZuVZ3A11rItvmk.jpeg
Qihao Liu
user
QHL067
666bb80de59c42693adae02b
Zhanpeng Zeng
admin_assigned
2024-06-14T09:51:20.683Z
646e2ddf4ad7f90727a0658b
/avatars/f62a9f835279b49ca5899c3aa6228dca.svg
Zhanpeng Zeng
user
mlpen
666bb80de59c42693adae02c
Ju He
claimed_verified
2024-11-04T09:36:37.337Z
661c9059bcd78151e5c06ea1
https://cdn-avatars.huggingface.co/v1/production/uploads/661c9059bcd78151e5c06ea1/27bfNo1LZeZQ77vWuAa10.png
Ju He
user
turkeyju
666bb80de59c42693adae02d
Qihang Yu
admin_assigned
2024-06-14T09:51:41.891Z
648010b00b9d0f49849adb19
/avatars/54419bb6e6ee788aba10cd64cd921204.svg
Qihang Yu
user
yucornetto
666bb80de59c42693adae02e
Xiaohui Shen
admin_assigned
2024-06-14T09:51:48.062Z
6430aa1b32a732121cd81f98
/avatars/5419f8d6d4d36fa5ac83e30667b9fd99.svg
Xiaohui Shen
user
XiaohuiShen
666bb80de59c42693adae02f
Liang-Chieh Chen
Alleviating Distortion in Image Generation via Multi-Resolution
  Diffusion Models
This paper presents innovative enhancements to diffusion models by
integrating a novel multi-resolution network and time-dependent layer
normalization. Diffusion models have gained prominence for their effectiveness
in high-fidelity image generation. While conventional approaches rely on
convolutional U-Net architectures, recent Transformer-based designs have
demonstrated superior performance and scalability. However, Transformer
architectures, which tokenize input data (via "patchification"), face a
trade-off between visual fidelity and computational complexity due to the
quadratic nature of self-attention operations concerning token length. While
larger patch sizes enable attention computation efficiency, they struggle to
capture fine-grained visual details, leading to image distortions. To address
this challenge, we propose augmenting the Diffusion model with the
Multi-Resolution network (DiMR), a framework that refines features across
multiple resolutions, progressively enhancing detail from low to high
resolution. Additionally, we introduce Time-Dependent Layer Normalization
(TD-LN), a parameter-efficient approach that incorporates time-dependent
parameters into layer normalization to inject time information and achieve
superior performance. Our method's efficacy is demonstrated on the
class-conditional ImageNet generation benchmark, where DiMR-XL variants
outperform prior diffusion models, setting new state-of-the-art FID scores of
1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512. Project page:
https://qihao067.github.io/projects/DiMR
A novel multi-resolution network and time-dependent layer normalization enhance diffusion models, improving image generation fidelity and efficiency.
diffusion models Multi-Resolution network DiMR time-dependent layer normalization TD-LN Transformer-based designs patchification self-attention visual fidelity high-resolution synthesis class-conditional generation ImageNet FID scores