Randomized Autoregressive Visual Generation
This paper presents Randomized AutoRegressive modeling (RAR) for visual
generation, which sets a new state-of-the-art performance on the image
generation task while maintaining full compatibility with language modeling
frameworks. The proposed RAR is simple: during a standard autoregressive
training process with a next-token prediction objective, the input
sequence-typically ordered in raster form-is randomly permuted into different
factorization orders with a probability r, where r starts at 1 and linearly
decays to 0 over the course of training. This annealing training strategy
enables the model to learn to maximize the expected likelihood over all
factorization orders and thus effectively improve the model's capability of
modeling bidirectional contexts. Importantly, RAR preserves the integrity of
the autoregressive modeling framework, ensuring full compatibility with
language modeling while significantly improving performance in image
generation. On the ImageNet-256 benchmark, RAR achieves an FID score of 1.48,
not only surpassing prior state-of-the-art autoregressive image generators but
also outperforming leading diffusion-based and masked transformer-based
methods. Code and models will be made available at
https://github.com/bytedance/1d-tokenizer
672837cb4c94449f57de011f
Qihang Yu
claimed_verified
2024-11-04T09:35:59.609Z
648010b00b9d0f49849adb19
/avatars/54419bb6e6ee788aba10cd64cd921204.svg
Qihang Yu
user
yucornetto
672837cb4c94449f57de0120
Ju He
claimed_verified
2024-11-04T09:36:06.564Z
661c9059bcd78151e5c06ea1
https://cdn-avatars.huggingface.co/v1/production/uploads/661c9059bcd78151e5c06ea1/27bfNo1LZeZQ77vWuAa10.png
Ju He
user
turkeyju
672837cb4c94449f57de0121
Xueqing Deng
admin_assigned
2024-11-04T14:24:03.091Z
65ca9b1743207e438a95e90c
/avatars/8f7bde1c44d8e665a29ee08ce7fedfa4.svg
Xueqing Deng
user
xdeng77
672837cb4c94449f57de0122
Xiaohui Shen
admin_assigned
2024-11-04T14:23:56.985Z
6430aa1b32a732121cd81f98
/avatars/5419f8d6d4d36fa5ac83e30667b9fd99.svg
Xiaohui Shen
user
XiaohuiShen
672837cb4c94449f57de0123
Liang-Chieh Chen
Randomized Autoregressive Visual Generation
This paper presents Randomized AutoRegressive modeling (RAR) for visual
generation, which sets a new state-of-the-art performance on the image
generation task while maintaining full compatibility with language modeling
frameworks. The proposed RAR is simple: during a standard autoregressive
training process with a next-token prediction objective, the input
sequence-typically ordered in raster form-is randomly permuted into different
factorization orders with a probability r, where r starts at 1 and linearly
decays to 0 over the course of training. This annealing training strategy
enables the model to learn to maximize the expected likelihood over all
factorization orders and thus effectively improve the model's capability of
modeling bidirectional contexts. Importantly, RAR preserves the integrity of
the autoregressive modeling framework, ensuring full compatibility with
language modeling while significantly improving performance in image
generation. On the ImageNet-256 benchmark, RAR achieves an FID score of 1.48,
not only surpassing prior state-of-the-art autoregressive image generators but
also outperforming leading diffusion-based and masked transformer-based
methods. Code and models will be made available at
https://github.com/bytedance/1d-tokenizer
Randomized AutoRegressive modeling (RAR) improves autoregressive image generation by randomly permuting input sequences during training, outperforming existing methods on the ImageNet-256 benchmark.
Randomized AutoRegressive modeling RAR autoregressive training next-token prediction factorization orders annealing training strategy bidirectional contexts FID score autoregressive image generators diffusion-based methods masked transformer-based methods