STARFlow: Scaling Latent Normalizing Flows for High-resolution Image
  Synthesis
We present STARFlow, a scalable generative model based on normalizing flows
that achieves strong performance in high-resolution image synthesis. The core
of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the
expressive power of normalizing flows with the structured modeling capabilities
of Autoregressive Transformers. We first establish the theoretical universality
of TARFlow for modeling continuous distributions. Building on this foundation,
we introduce several key architectural and algorithmic innovations to
significantly enhance scalability: (1) a deep-shallow design, wherein a deep
Transformer block captures most of the model representational capacity,
complemented by a few shallow Transformer blocks that are computationally
efficient yet substantially beneficial; (2) modeling in the latent space of
pretrained autoencoders, which proves more effective than direct pixel-level
modeling; and (3) a novel guidance algorithm that significantly boosts sample
quality. Crucially, our model remains an end-to-end normalizing flow, enabling
exact maximum likelihood training in continuous spaces without discretization.
STARFlow achieves competitive performance in both class-conditional and
text-conditional image generation tasks, approaching state-of-the-art diffusion
models in sample quality. To our knowledge, this work is the first successful
demonstration of normalizing flows operating effectively at this scale and
resolution.
68466dfb3ec10bdd8ab4dae2
Jiatao Gu
68466dfb3ec10bdd8ab4dae3
Tianrong Chen
68466dfb3ec10bdd8ab4dae4
David Berthelot
68466dfb3ec10bdd8ab4dae5
Huangjie Zheng
68466dfb3ec10bdd8ab4dae6
Yuyang Wang
68466dfb3ec10bdd8ab4dae7
Ruixiang Zhang
68466dfb3ec10bdd8ab4dae8
Laurent Dinh
68466dfb3ec10bdd8ab4dae9
Miguel Angel Bautista
68466dfb3ec10bdd8ab4daea
Josh Susskind
68466dfb3ec10bdd8ab4daeb
Shuangfei Zhai
STARFlow: Scaling Latent Normalizing Flows for High-resolution Image
  Synthesis
We present STARFlow, a scalable generative model based on normalizing flows
that achieves strong performance in high-resolution image synthesis. The core
of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the
expressive power of normalizing flows with the structured modeling capabilities
of Autoregressive Transformers. We first establish the theoretical universality
of TARFlow for modeling continuous distributions. Building on this foundation,
we introduce several key architectural and algorithmic innovations to
significantly enhance scalability: (1) a deep-shallow design, wherein a deep
Transformer block captures most of the model representational capacity,
complemented by a few shallow Transformer blocks that are computationally
efficient yet substantially beneficial; (2) modeling in the latent space of
pretrained autoencoders, which proves more effective than direct pixel-level
modeling; and (3) a novel guidance algorithm that significantly boosts sample
quality. Crucially, our model remains an end-to-end normalizing flow, enabling
exact maximum likelihood training in continuous spaces without discretization.
STARFlow achieves competitive performance in both class-conditional and
text-conditional image generation tasks, approaching state-of-the-art diffusion
models in sample quality. To our knowledge, this work is the first successful
demonstration of normalizing flows operating effectively at this scale and
resolution.
STARFlow, a generative model combining normalizing flows with autoregressive Transformers, achieves competitive image synthesis performance with innovations in architecture and latent space modeling.
normalizing flows Transformer Autoregressive Flow TARFlow theoretical universality deep-shallow design pretrained autoencoders latent space guidance algorithm end-to-end normalizing flow exact maximum likelihood training class-conditional text-conditional image generation state-of-the-art diffusion models