Understand Before You Generate: Self-Guided Training for Autoregressive
  Image Generation
Recent studies have demonstrated the importance of high-quality visual
representations in image generation and have highlighted the limitations of
generative models in image understanding. As a generative paradigm originally
designed for natural language, autoregressive models face similar challenges.
In this work, we present the first systematic investigation into the mechanisms
of applying the next-token prediction paradigm to the visual domain. We
identify three key properties that hinder the learning of high-level visual
semantics: local and conditional dependence, inter-step semantic inconsistency,
and spatial invariance deficiency. We show that these issues can be effectively
addressed by introducing self-supervised objectives during training, leading to
a novel training framework, Self-guided Training for AutoRegressive models
(ST-AR). Without relying on pre-trained representation models, ST-AR
significantly enhances the image understanding ability of autoregressive models
and leads to improved generation quality. Specifically, ST-AR brings
approximately 42% FID improvement for LlamaGen-L and 49% FID improvement for
LlamaGen-XL, while maintaining the same sampling strategy.
68ccc7243df9ac65e93dc679
Xiaoyu Yue
claimed_verified
2025-09-19T06:48:19.187Z
6662a450b1fff5575fdf0fbd
/avatars/2a6065269f1980213625a9cfd8d42fbd.svg
Xiaoyu Yue
user
YueXY233
68ccc7243df9ac65e93dc67a
Zidong Wang
claimed_verified
2025-09-19T07:00:14.082Z
64b7aa374df206a3ed1947d2
https://cdn-avatars.huggingface.co/v1/production/uploads/64b7aa374df206a3ed1947d2/Ostk72ehOR6yUX-PhUvyQ.jpeg
wzd
user
GoodEnough
68ccc7243df9ac65e93dc67b
Yuqing Wang
68ccc7243df9ac65e93dc67c
Wenlong Zhang
68ccc7243df9ac65e93dc67d
Xihui Liu
68ccc7243df9ac65e93dc67e
Wanli Ouyang
68ccc7243df9ac65e93dc67f
Lei Bai
68ccc7243df9ac65e93dc680
Luping Zhou
Understand Before You Generate: Self-Guided Training for Autoregressive
  Image Generation
Recent studies have demonstrated the importance of high-quality visual
representations in image generation and have highlighted the limitations of
generative models in image understanding. As a generative paradigm originally
designed for natural language, autoregressive models face similar challenges.
In this work, we present the first systematic investigation into the mechanisms
of applying the next-token prediction paradigm to the visual domain. We
identify three key properties that hinder the learning of high-level visual
semantics: local and conditional dependence, inter-step semantic inconsistency,
and spatial invariance deficiency. We show that these issues can be effectively
addressed by introducing self-supervised objectives during training, leading to
a novel training framework, Self-guided Training for AutoRegressive models
(ST-AR). Without relying on pre-trained representation models, ST-AR
significantly enhances the image understanding ability of autoregressive models
and leads to improved generation quality. Specifically, ST-AR brings
approximately 42% FID improvement for LlamaGen-L and 49% FID improvement for
LlamaGen-XL, while maintaining the same sampling strategy.
Self-guided Training for AutoRegressive models (ST-AR) enhances image understanding and generation quality in autoregressive models by addressing key visual semantics challenges through self-supervised objectives.
autoregressive models next-token prediction high-level visual semantics local and conditional dependence inter-step semantic inconsistency spatial invariance deficiency self-supervised objectives Self-guided Training for AutoRegressive models ST-AR FID improvement LlamaGen-L LlamaGen-XL