Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model
Diffusion-based image generation models excel at producing high-quality
synthetic content, but suffer from slow and computationally expensive
inference. Prior work has attempted to mitigate this by caching and reusing
features within diffusion transformers across inference steps. These methods,
however, often rely on rigid heuristics that result in limited acceleration or
poor generalization across architectures. We propose Evolutionary Caching to
Accelerate Diffusion models (ECAD), a genetic algorithm that learns efficient,
per-model, caching schedules forming a Pareto frontier, using only a small set
of calibration prompts. ECAD requires no modifications to network parameters or
reference images. It offers significant inference speedups, enables
fine-grained control over the quality-latency trade-off, and adapts seamlessly
to different diffusion models. Notably, ECAD's learned schedules can generalize
effectively to resolutions and model variants not seen during calibration. We
evaluate ECAD on PixArt-alpha, PixArt-Sigma, and FLUX-1.dev using multiple
metrics (FID, CLIP, Image Reward) across diverse benchmarks (COCO, MJHQ-30k,
PartiPrompts), demonstrating consistent improvements over previous approaches.
On PixArt-alpha, ECAD identifies a schedule that outperforms the previous
state-of-the-art method by 4.47 COCO FID while increasing inference speedup
from 2.35x to 2.58x. Our results establish ECAD as a scalable and generalizable
approach for accelerating diffusion inference. Our project website is available
at https://aniaggarwal.github.io/ecad and our code is available at
https://github.com/aniaggarwal/ecad.
68544c287bc8d012d4ca9902
Anirud Aggarwal
claimed_verified
2025-07-03T07:41:48.238Z
67d5e12a49b082fa5174cc9e
/avatars/a7a5299bb70cb6b0e9ac34e6279d4702.svg
Ani Aggarwal
user
AniAggarwal
68544c287bc8d012d4ca9903
Abhinav Shrivastava
68544c287bc8d012d4ca9904
Matthew Gwilliam
Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model
Diffusion-based image generation models excel at producing high-quality
synthetic content, but suffer from slow and computationally expensive
inference. Prior work has attempted to mitigate this by caching and reusing
features within diffusion transformers across inference steps. These methods,
however, often rely on rigid heuristics that result in limited acceleration or
poor generalization across architectures. We propose Evolutionary Caching to
Accelerate Diffusion models (ECAD), a genetic algorithm that learns efficient,
per-model, caching schedules forming a Pareto frontier, using only a small set
of calibration prompts. ECAD requires no modifications to network parameters or
reference images. It offers significant inference speedups, enables
fine-grained control over the quality-latency trade-off, and adapts seamlessly
to different diffusion models. Notably, ECAD's learned schedules can generalize
effectively to resolutions and model variants not seen during calibration. We
evaluate ECAD on PixArt-alpha, PixArt-Sigma, and FLUX-1.dev using multiple
metrics (FID, CLIP, Image Reward) across diverse benchmarks (COCO, MJHQ-30k,
PartiPrompts), demonstrating consistent improvements over previous approaches.
On PixArt-alpha, ECAD identifies a schedule that outperforms the previous
state-of-the-art method by 4.47 COCO FID while increasing inference speedup
from 2.35x to 2.58x. Our results establish ECAD as a scalable and generalizable
approach for accelerating diffusion inference. Our project website is available
at https://aniaggarwal.github.io/ecad and our code is available at
https://github.com/aniaggarwal/ecad.
ECAD, a genetic algorithm, optimizes caching schedules for diffusion models, enhancing inference speed while maintaining quality across various benchmarks.
diffusion-based image generation diffusion transformers evolutionary caching genetic algorithm caching schedules inference speedups quality-latency trade-off calibration prompts PixArt-alpha PixArt-Sigma FLUX-1.dev FID CLIP Image Reward COCO MJHQ-30k PartiPrompts