Neighboring Autoregressive Modeling for Efficient Visual Generation
Visual autoregressive models typically adhere to a raster-order ``next-token
prediction" paradigm, which overlooks the spatial and temporal locality
inherent in visual content. Specifically, visual tokens exhibit significantly
stronger correlations with their spatially or temporally adjacent tokens
compared to those that are distant. In this paper, we propose Neighboring
Autoregressive Modeling (NAR), a novel paradigm that formulates autoregressive
visual generation as a progressive outpainting procedure, following a
near-to-far ``next-neighbor prediction" mechanism. Starting from an initial
token, the remaining tokens are decoded in ascending order of their Manhattan
distance from the initial token in the spatial-temporal space, progressively
expanding the boundary of the decoded region. To enable parallel prediction of
multiple adjacent tokens in the spatial-temporal space, we introduce a set of
dimension-oriented decoding heads, each predicting the next token along a
mutually orthogonal dimension. During inference, all tokens adjacent to the
decoded tokens are processed in parallel, substantially reducing the model
forward steps for generation. Experiments on ImageNet256times 256 and UCF101
demonstrate that NAR achieves 2.4times and 8.6times higher throughput
respectively, while obtaining superior FID/FVD scores for both image and video
generation tasks compared to the PAR-4X approach. When evaluating on
text-to-image generation benchmark GenEval, NAR with 0.8B parameters
outperforms Chameleon-7B while using merely 0.4 of the training data. Code is
available at https://github.com/ThisisBillhe/NAR.
67d7e9bd93b8599318993db2
Yefei He
admin_assigned
2025-03-17T14:54:40.739Z
65a88c3d26598b995531fff1
/avatars/b1a524857d8572d0405476661b434160.svg
Yefei He
user
yefly
67d7e9bd93b8599318993db3
Yuanyu He
admin_assigned
2025-03-17T14:55:03.703Z
668a94d30b04331a0b6dc457
/avatars/66a311a7474e9d297231868dbbd6a18c.svg
heyuanyu
user
heyuanyu
67d7e9bd93b8599318993db4
Shaoxuan He
admin_assigned
2025-03-17T14:54:33.115Z
66365ad3ea6f5788b32a2fb4
/avatars/99a7aebc1d022b52d7692494bdd2a1c7.svg
Shaoxuan He
user
shxhe
67d7e9bd93b8599318993db5
Feng Chen
claimed_verified
2025-03-17T14:42:15.701Z
64574380f3ef144c0e69d484
/avatars/a0a84757cb0bf09c24291803e1389b49.svg
Feng Chen
user
chenfeng1271
67d7e9bd93b8599318993db6
Hong Zhou
67d7e9bd93b8599318993db7
Kaipeng Zhang
claimed_verified
2025-06-02T07:52:25.151Z
65f1713552c38a91e0a445e8
/avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
kaipeng
user
kpzhang996
67d7e9bd93b8599318993db8
Bohan Zhuang
admin_assigned
2025-03-17T14:55:23.366Z
65a8a828ef14f9e6037dfad2
/avatars/55fdc0393b2e37ac998ddbdb1e1ff636.svg
Bohan Zhuang
user
BohanZ
Neighboring Autoregressive Modeling for Efficient Visual Generation
Visual autoregressive models typically adhere to a raster-order ``next-token
prediction" paradigm, which overlooks the spatial and temporal locality
inherent in visual content. Specifically, visual tokens exhibit significantly
stronger correlations with their spatially or temporally adjacent tokens
compared to those that are distant. In this paper, we propose Neighboring
Autoregressive Modeling (NAR), a novel paradigm that formulates autoregressive
visual generation as a progressive outpainting procedure, following a
near-to-far ``next-neighbor prediction" mechanism. Starting from an initial
token, the remaining tokens are decoded in ascending order of their Manhattan
distance from the initial token in the spatial-temporal space, progressively
expanding the boundary of the decoded region. To enable parallel prediction of
multiple adjacent tokens in the spatial-temporal space, we introduce a set of
dimension-oriented decoding heads, each predicting the next token along a
mutually orthogonal dimension. During inference, all tokens adjacent to the
decoded tokens are processed in parallel, substantially reducing the model
forward steps for generation. Experiments on ImageNet256times 256 and UCF101
demonstrate that NAR achieves 2.4times and 8.6times higher throughput
respectively, while obtaining superior FID/FVD scores for both image and video
generation tasks compared to the PAR-4X approach. When evaluating on
text-to-image generation benchmark GenEval, NAR with 0.8B parameters
outperforms Chameleon-7B while using merely 0.4 of the training data. Code is
available at https://github.com/ThisisBillhe/NAR.
Neighboring Autoregressive Modeling (NAR) improves visual generation throughput and quality by predicting tokens in a near-to-far spatial-temporal order and using dimension-oriented decoding heads.
visual autoregressive models next-token prediction spatial locality temporal locality visual tokens Neighboring Autoregressive Modeling NAR progressive outpainting Manhattan distance spatial-temporal space dimension-oriented decoding heads FID scores FVD scores text-to-image generation GenEval Chameleon-7B