Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation
Zhi-Kai Chen, Jun-Peng Jiang, Han-Jia Ye, De-Chuan Zhan
Autoregressive (AR) image generation models can produce high-fidelity images but often struggle with slow inference due to their token-by-token, sequential decoding. Speculative decoding, which employs a draft model to approximate the AR model’s output, offers a promising way to reduce inference time. While this technique has been successfully applied to accelerate text-based AR models without sacrificing output quality, its application to image generation remains largely unexplored. Directly adapting this method to images is challenging because of the substantially larger sampling space, which complicates alignment between speculative and target model predictions, and the inadequate use of two-dimensional spatial information, which limits the exploitation of local image dependencies. To address these obstacles, we propose Spatial Speculative Decoding, a novel approach that leverages the inherent two-dimensional structure of images to guide a speculative model toward more accurate predictions and faster token generation. Experimental results on multiple text-to-image benchmarks demonstrate a 1.71× speedup over standard AR models, while preserving both image fidelity and diversity.