SiT: Exploring Flow and Diffusion-based Generative Models with Scalable
  Interpolant Transformers
We present Scalable Interpolant Transformers (SiT), a family of generative
models built on the backbone of Diffusion Transformers (DiT). The interpolant
framework, which allows for connecting two distributions in a more flexible way
than standard diffusion models, makes possible a modular study of various
design choices impacting generative models built on dynamical transport: using
discrete vs. continuous time learning, deciding the objective for the model to
learn, choosing the interpolant connecting the distributions, and deploying a
deterministic or stochastic sampler. By carefully introducing the above
ingredients, SiT surpasses DiT uniformly across model sizes on the conditional
ImageNet 256x256 benchmark using the exact same backbone, number of parameters,
and GFLOPs. By exploring various diffusion coefficients, which can be tuned
separately from learning, SiT achieves an FID-50K score of 2.06.
65a8afd76cb12bdc272f2774
Nanye Ma
65a8afd76cb12bdc272f2775
Mark Goldstein
admin_assigned
2024-01-18T09:57:45.428Z
6470f300d9360cd9d8e9219b
/avatars/dc09260e9912d7e626cad5c670ec8d3f.svg
mark goldstein
user
marikgoldstein
65a8afd76cb12bdc272f2776
Michael S. Albergo
admin_assigned
2024-01-18T09:58:12.428Z
636fbc0731af06da864cd7eb
/avatars/415db2ff8f79440a0216dd39cf6d0d43.svg
Michael Albergo
user
malbergo
65a8afd76cb12bdc272f2777
Nicholas M. Boffi
claimed_verified
2024-01-23T17:09:22.554Z
65afd8b7bd418aab3a677782
/avatars/11aa33b32ef4b6aa27ab8e6c068afa14.svg
Nicholas Boffi
user
nmboffi
65a8afd76cb12bdc272f2778
Eric Vanden-Eijnden
65a8afd76cb12bdc272f2779
Saining Xie
admin_assigned
2024-01-18T09:58:57.348Z
6596422646624a86ff3b3bda
/avatars/216e12b77e45ac5f1fa20932f5745411.svg
Saining Xie
user
sainx
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable
  Interpolant Transformers
We present Scalable Interpolant Transformers (SiT), a family of generative
models built on the backbone of Diffusion Transformers (DiT). The interpolant
framework, which allows for connecting two distributions in a more flexible way
than standard diffusion models, makes possible a modular study of various
design choices impacting generative models built on dynamical transport: using
discrete vs. continuous time learning, deciding the objective for the model to
learn, choosing the interpolant connecting the distributions, and deploying a
deterministic or stochastic sampler. By carefully introducing the above
ingredients, SiT surpasses DiT uniformly across model sizes on the conditional
ImageNet 256x256 benchmark using the exact same backbone, number of parameters,
and GFLOPs. By exploring various diffusion coefficients, which can be tuned
separately from learning, SiT achieves an FID-50K score of 2.06.
Scalable Interpolant Transformers (SiT), built on Diffusion Transformers (DiT), enhance generative models through a flexible interpolant framework, achieving superior performance on ImageNet 256x256 with tunable diffusion coefficients.
Scalable Interpolant Transformers SiT Diffusion Transformers DiT interpolant framework dynamical transport discrete time learning continuous time learning stochastic sampler diffusion coefficients FID-50K score