Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive
  Text-to-image Generation
As a new paradigm of visual content generation, autoregressive text-to-image
models suffer from slow inference due to their sequential token-by-token
decoding process, often requiring thousands of model forward passes to generate
a single image. To address this inefficiency, we propose Speculative
Jacobi-Denoising Decoding (SJD2), a framework that incorporates the denoising
process into Jacobi iterations to enable parallel token generation in
autoregressive models. Our method introduces a next-clean-token prediction
paradigm that enables the pre-trained autoregressive models to accept
noise-perturbed token embeddings and predict the next clean tokens through
low-cost fine-tuning. This denoising paradigm guides the model towards more
stable Jacobi trajectories. During inference, our method initializes token
sequences with Gaussian noise and performs iterative
next-clean-token-prediction in the embedding space. We employ a probabilistic
criterion to verify and accept multiple tokens in parallel, and refine the
unaccepted tokens for the next iteration with the denoising trajectory.
Experiments show that our method can accelerate generation by reducing model
forward passes while maintaining the visual quality of generated images.
68ec791ccd07fb414898ca2e
Yao Teng
68ec791ccd07fb414898ca2f
Fuyun Wang
68ec791ccd07fb414898ca30
Xian Liu
68ec791ccd07fb414898ca31
Zhekai Chen
68ec791ccd07fb414898ca32
Han Shi
68ec791ccd07fb414898ca33
Yu Wang
68ec791ccd07fb414898ca34
Zhenguo Li
68ec791ccd07fb414898ca35
Weiyang Liu
68ec791ccd07fb414898ca36
Difan Zou
68ec791ccd07fb414898ca37
Xihui Liu
claimed_verified
2025-10-14T07:32:01.549Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive
  Text-to-image Generation
As a new paradigm of visual content generation, autoregressive text-to-image
models suffer from slow inference due to their sequential token-by-token
decoding process, often requiring thousands of model forward passes to generate
a single image. To address this inefficiency, we propose Speculative
Jacobi-Denoising Decoding (SJD2), a framework that incorporates the denoising
process into Jacobi iterations to enable parallel token generation in
autoregressive models. Our method introduces a next-clean-token prediction
paradigm that enables the pre-trained autoregressive models to accept
noise-perturbed token embeddings and predict the next clean tokens through
low-cost fine-tuning. This denoising paradigm guides the model towards more
stable Jacobi trajectories. During inference, our method initializes token
sequences with Gaussian noise and performs iterative
next-clean-token-prediction in the embedding space. We employ a probabilistic
criterion to verify and accept multiple tokens in parallel, and refine the
unaccepted tokens for the next iteration with the denoising trajectory.
Experiments show that our method can accelerate generation by reducing model
forward passes while maintaining the visual quality of generated images.
Speculative Jacobi-Denoising Decoding accelerates autoregressive text-to-image generation by enabling parallel token prediction and reducing model forward passes.
autoregressive text-to-image models Speculative Jacobi-Denoising Decoding denoising process Jacobi iterations next-clean-token prediction token embeddings low-cost fine-tuning Gaussian noise next-clean-token-prediction probabilistic criterion visual quality