FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality
  Samples with Less Compute
Despite their remarkable performance, modern Diffusion Transformers are
hindered by substantial resource requirements during inference, stemming from
the fixed and large amount of compute needed for each denoising step. In this
work, we revisit the conventional static paradigm that allocates a fixed
compute budget per denoising iteration and propose a dynamic strategy instead.
Our simple and sample-efficient framework enables pre-trained DiT models to be
converted into flexible ones -- dubbed FlexiDiT -- allowing them to
process inputs at varying compute budgets. We demonstrate how a single
flexible model can generate images without any drop in quality, while
reducing the required FLOPs by more than 40\% compared to their static
counterparts, for both class-conditioned and text-conditioned image generation.
Our method is general and agnostic to input and conditioning modalities. We
show how our approach can be readily extended for video generation, where
FlexiDiT models generate samples with up to 75\% less compute without
compromising performance.
67c14524af5eaa8dd062a216
Sotiris Anagnostidis
admin_assigned
2025-02-28T12:32:18.844Z
62f8f4ff92e64c61bc6938da
/avatars/d386eb35d2c3d52186b2a8ec957f51bc.svg
Sotiris Anagnostidis
user
sanagnos
67c14524af5eaa8dd062a217
Gregor Bachmann
admin_assigned
2025-02-28T12:32:24.781Z
64996389a3f227b05cbd956f
/avatars/a586e7f99efdc7b61e05d62945575096.svg
Gregor Bachmann
user
gregorbachmann
67c14524af5eaa8dd062a218
Yeongmin Kim
admin_assigned
2025-02-28T12:32:30.708Z
65d6a010c57d1c140e395e31
/avatars/ddc9cfc98da36b639bd9205ee65b6967.svg
Yeongmin Kim
user
YeongminKim
67c14524af5eaa8dd062a219
Jonas Kohler
admin_assigned
2025-02-28T12:32:36.925Z
6408a3a19e9f790c905281c2
/avatars/3517962e54bda141018e13f7e21fb1ae.svg
jonas köhler
user
Jonaskohler
67c14524af5eaa8dd062a21a
Markos Georgopoulos
67c14524af5eaa8dd062a21b
Artsiom Sanakoyeu
67c14524af5eaa8dd062a21c
Yuming Du
67c14524af5eaa8dd062a21d
Albert Pumarola
67c14524af5eaa8dd062a21e
Ali Thabet
67c14524af5eaa8dd062a21f
Edgar Schönfeld
claimed_verified
2025-03-02T20:18:03.571Z
6571c662d5c6a6d3b0bdae88
/avatars/8080bcdf8b331f62383b724050189660.svg
Edgar Schoenfeld
user
edgarschoenfeld
FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality
  Samples with Less Compute
Despite their remarkable performance, modern Diffusion Transformers are
hindered by substantial resource requirements during inference, stemming from
the fixed and large amount of compute needed for each denoising step. In this
work, we revisit the conventional static paradigm that allocates a fixed
compute budget per denoising iteration and propose a dynamic strategy instead.
Our simple and sample-efficient framework enables pre-trained DiT models to be
converted into flexible ones -- dubbed FlexiDiT -- allowing them to
process inputs at varying compute budgets. We demonstrate how a single
flexible model can generate images without any drop in quality, while
reducing the required FLOPs by more than 40\% compared to their static
counterparts, for both class-conditioned and text-conditioned image generation.
Our method is general and agnostic to input and conditioning modalities. We
show how our approach can be readily extended for video generation, where
FlexiDiT models generate samples with up to 75\% less compute without
compromising performance.
A flexible diffusion transformer model reduces computational requirements for image and video generation without sacrificing quality.
Diffusion Transformers denoising steps compute budget flexible models FLOPs class-conditioned text-conditioned video generation