DeepCache: Accelerating Diffusion Models for Free
Diffusion models have recently gained unprecedented attention in the field of
image synthesis due to their remarkable generative capabilities.
Notwithstanding their prowess, these models often incur substantial
computational costs, primarily attributed to the sequential denoising process
and cumbersome model size. Traditional methods for compressing diffusion models
typically involve extensive retraining, presenting cost and feasibility
challenges. In this paper, we introduce DeepCache, a novel training-free
paradigm that accelerates diffusion models from the perspective of model
architecture. DeepCache capitalizes on the inherent temporal redundancy
observed in the sequential denoising steps of diffusion models, which caches
and retrieves features across adjacent denoising stages, thereby curtailing
redundant computations. Utilizing the property of the U-Net, we reuse the
high-level features while updating the low-level features in a very cheap way.
This innovative strategy, in turn, enables a speedup factor of 2.3times for
Stable Diffusion v1.5 with only a 0.05 decline in CLIP Score, and 4.1times
for LDM-4-G with a slight decrease of 0.22 in FID on ImageNet. Our experiments
also demonstrate DeepCache's superiority over existing pruning and distillation
methods that necessitate retraining and its compatibility with current sampling
techniques. Furthermore, we find that under the same throughput, DeepCache
effectively achieves comparable or even marginally improved results with DDIM
or PLMS. The code is available at https://github.com/horseee/DeepCache
656ea1e3228bbe9eb46c76e4
Xinyin Ma
claimed_verified
2023-12-05T08:39:01.637Z
64396ebc21221ac7411852b3
https://cdn-avatars.huggingface.co/v1/production/uploads/64396ebc21221ac7411852b3/SR0dC8N0bdj9tZFxYPpSf.jpeg
Xinyin Ma
user
horseee
656ea1e3228bbe9eb46c76e5
Gongfan Fang
656ea1e3228bbe9eb46c76e6
Xinchao Wang
admin_assigned
2023-12-05T15:08:52.796Z
63fc03a50aab060792ffef39
/avatars/9d5b1bb2a41928e08176b703935133ab.svg
Wangxinchao
user
wxcTest
DeepCache: Accelerating Diffusion Models for Free
Diffusion models have recently gained unprecedented attention in the field of
image synthesis due to their remarkable generative capabilities.
Notwithstanding their prowess, these models often incur substantial
computational costs, primarily attributed to the sequential denoising process
and cumbersome model size. Traditional methods for compressing diffusion models
typically involve extensive retraining, presenting cost and feasibility
challenges. In this paper, we introduce DeepCache, a novel training-free
paradigm that accelerates diffusion models from the perspective of model
architecture. DeepCache capitalizes on the inherent temporal redundancy
observed in the sequential denoising steps of diffusion models, which caches
and retrieves features across adjacent denoising stages, thereby curtailing
redundant computations. Utilizing the property of the U-Net, we reuse the
high-level features while updating the low-level features in a very cheap way.
This innovative strategy, in turn, enables a speedup factor of 2.3times for
Stable Diffusion v1.5 with only a 0.05 decline in CLIP Score, and 4.1times
for LDM-4-G with a slight decrease of 0.22 in FID on ImageNet. Our experiments
also demonstrate DeepCache's superiority over existing pruning and distillation
methods that necessitate retraining and its compatibility with current sampling
techniques. Furthermore, we find that under the same throughput, DeepCache
effectively achieves comparable or even marginally improved results with DDIM
or PLMS. The code is available at https://github.com/horseee/DeepCache
DeepCache, a training-free method, accelerates diffusion models by caching and reusing features across denoising stages, achieving significant speedups with minimal quality degradation.
diffusion models sequential denoising DeepCache U-Net feature caching stable diffusion LDM CLIP Score FID pruning distillation DDIM PLMS