DiSA: Diffusion Step Annealing in Autoregressive Image Generation
An increasing number of autoregressive models, such as MAR, FlowAR, xAR, and
Harmon adopt diffusion sampling to improve the quality of image generation.
However, this strategy leads to low inference efficiency, because it usually
takes 50 to 100 steps for diffusion to sample a token. This paper explores how
to effectively address this issue. Our key motivation is that as more tokens
are generated during the autoregressive process, subsequent tokens follow more
constrained distributions and are easier to sample. To intuitively explain, if
a model has generated part of a dog, the remaining tokens must complete the dog
and thus are more constrained. Empirical evidence supports our motivation: at
later generation stages, the next tokens can be well predicted by a multilayer
perceptron, exhibit low variance, and follow closer-to-straight-line denoising
paths from noise to tokens. Based on our finding, we introduce diffusion step
annealing (DiSA), a training-free method which gradually uses fewer diffusion
steps as more tokens are generated, e.g., using 50 steps at the beginning and
gradually decreasing to 5 steps at later stages. Because DiSA is derived from
our finding specific to diffusion in autoregressive models, it is complementary
to existing acceleration methods designed for diffusion alone. DiSA can be
implemented in only a few lines of code on existing models, and albeit simple,
achieves 5-10times faster inference for MAR and Harmon and 1.4-2.5times
for FlowAR and xAR, while maintaining the generation quality.
68364bbf8bb68ffc201e7d83
Qinyu Zhao
claimed_verified
2025-05-28T08:58:34.882Z
6503e36206e7b98952fa9052
/avatars/a8a57683e9e3915d9833d3b9512c1ba0.svg
Qinyu Zhao
user
QinyuZhao1116
68364bbf8bb68ffc201e7d84
Jaskirat Singh
68364bbf8bb68ffc201e7d85
Ming Xu
68364bbf8bb68ffc201e7d86
Akshay Asthana
68364bbf8bb68ffc201e7d87
Stephen Gould
68364bbf8bb68ffc201e7d88
Liang Zheng
DiSA: Diffusion Step Annealing in Autoregressive Image Generation
An increasing number of autoregressive models, such as MAR, FlowAR, xAR, and
Harmon adopt diffusion sampling to improve the quality of image generation.
However, this strategy leads to low inference efficiency, because it usually
takes 50 to 100 steps for diffusion to sample a token. This paper explores how
to effectively address this issue. Our key motivation is that as more tokens
are generated during the autoregressive process, subsequent tokens follow more
constrained distributions and are easier to sample. To intuitively explain, if
a model has generated part of a dog, the remaining tokens must complete the dog
and thus are more constrained. Empirical evidence supports our motivation: at
later generation stages, the next tokens can be well predicted by a multilayer
perceptron, exhibit low variance, and follow closer-to-straight-line denoising
paths from noise to tokens. Based on our finding, we introduce diffusion step
annealing (DiSA), a training-free method which gradually uses fewer diffusion
steps as more tokens are generated, e.g., using 50 steps at the beginning and
gradually decreasing to 5 steps at later stages. Because DiSA is derived from
our finding specific to diffusion in autoregressive models, it is complementary
to existing acceleration methods designed for diffusion alone. DiSA can be
implemented in only a few lines of code on existing models, and albeit simple,
achieves 5-10times faster inference for MAR and Harmon and 1.4-2.5times
for FlowAR and xAR, while maintaining the generation quality.
Diffusion step annealing enhances inference efficiency in autoregressive models by reducing the number of diffusion steps as more tokens are generated, preserving quality.
autoregressive models MAR FlowAR xAR Harmon diffusion sampling diffusion step annealing DiSA multilayer perceptron denoising paths