Continuous Speculative Decoding for Autoregressive Image Generation
Continuous-valued Autoregressive (AR) image generation models have
demonstrated notable superiority over their discrete-token counterparts,
showcasing considerable reconstruction quality and higher generation fidelity.
However, the computational demands of the autoregressive framework result in
significant inference overhead. While speculative decoding has proven effective
in accelerating Large Language Models (LLMs), their adaptation to
continuous-valued visual autoregressive models remains unexplored. This work
generalizes the speculative decoding algorithm from discrete tokens to
continuous space. By analyzing the intrinsic properties of output distribution,
we establish a tailored acceptance criterion for the diffusion distributions
prevalent in such models. To overcome the inconsistency that occurred in
speculative decoding output distributions, we introduce denoising trajectory
alignment and token pre-filling methods. Additionally, we identify the
hard-to-sample distribution in the rejection phase. To mitigate this issue, we
propose a meticulous acceptance-rejection sampling method with a proper upper
bound, thereby circumventing complex integration. Experimental results show
that our continuous speculative decoding achieves a remarkable 2.33times
speed-up on off-the-shelf models while maintaining the output distribution.
Codes will be available at https://github.com/MarkXCloud/CSpD
673d4d484bec167076968b8c
Zili Wang
claimed_verified
2024-11-20T10:08:39.427Z
6410998db27543634e3aacc3
https://cdn-avatars.huggingface.co/v1/production/uploads/1678809388600-noauth.png
Zili Wang
user
MarkWang
673d4d484bec167076968b8d
Robert Zhang
673d4d484bec167076968b8e
Kun Ding
673d4d484bec167076968b8f
Qi Yang
claimed_verified
2025-06-10T09:31:47.075Z
643e45c4c639f8bc9727810a
https://cdn-avatars.huggingface.co/v1/production/uploads/643e45c4c639f8bc9727810a/BJR1cvSCxcqxr08iS7GsI.jpeg
YannQi
user
YannQi
673d4d484bec167076968b90
Fei Li
673d4d484bec167076968b91
Shiming Xiang
Continuous Speculative Decoding for Autoregressive Image Generation
Continuous-valued Autoregressive (AR) image generation models have
demonstrated notable superiority over their discrete-token counterparts,
showcasing considerable reconstruction quality and higher generation fidelity.
However, the computational demands of the autoregressive framework result in
significant inference overhead. While speculative decoding has proven effective
in accelerating Large Language Models (LLMs), their adaptation to
continuous-valued visual autoregressive models remains unexplored. This work
generalizes the speculative decoding algorithm from discrete tokens to
continuous space. By analyzing the intrinsic properties of output distribution,
we establish a tailored acceptance criterion for the diffusion distributions
prevalent in such models. To overcome the inconsistency that occurred in
speculative decoding output distributions, we introduce denoising trajectory
alignment and token pre-filling methods. Additionally, we identify the
hard-to-sample distribution in the rejection phase. To mitigate this issue, we
propose a meticulous acceptance-rejection sampling method with a proper upper
bound, thereby circumventing complex integration. Experimental results show
that our continuous speculative decoding achieves a remarkable 2.33times
speed-up on off-the-shelf models while maintaining the output distribution.
Codes will be available at https://github.com/MarkXCloud/CSpD
A speculative decoding algorithm is adapted for continuous-valued image generation to reduce inference time while preserving output quality.
Continue-valued Autoregressive models speculative decoding Large Language Models diffusion distributions denoising trajectory alignment token pre-filling acceptance-rejection sampling