X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image
  Generative Models Great Again
Numerous efforts have been made to extend the ``next token prediction''
paradigm to visual contents, aiming to create a unified approach for both image
generation and understanding. Nevertheless, attempts to generate images through
autoregressive modeling with discrete tokens have been plagued by issues such
as low visual fidelity, distorted outputs, and failure to adhere to complex
instructions when rendering intricate details. These shortcomings are likely
attributed to cumulative errors during autoregressive inference or information
loss incurred during the discretization process. Probably due to this
challenge, recent research has increasingly shifted toward jointly training
image generation with diffusion objectives and language generation with
autoregressive objectives, moving away from unified modeling approaches. In
this work, we demonstrate that reinforcement learning can effectively mitigate
artifacts and largely enhance the generation quality of a discrete
autoregressive modeling method, thereby enabling seamless integration of image
and language generation. Our framework comprises a semantic image tokenizer, a
unified autoregressive model for both language and images, and an offline
diffusion decoder for image generation, termed X-Omni. X-Omni achieves
state-of-the-art performance in image generation tasks using a 7B language
model, producing images with high aesthetic quality while exhibiting strong
capabilities in following instructions and rendering long texts.
6889746631e1218a08928182
Zigang Geng
6889746631e1218a08928183
Yibing Wang
6889746631e1218a08928184
Yeyao Ma
6889746631e1218a08928185
Chen Li
6889746631e1218a08928186
Yongming Rao
6889746631e1218a08928187
Shuyang Gu
6889746631e1218a08928188
Zhao Zhong
6889746631e1218a08928189
Qinglin Lu
6889746631e1218a0892818a
Han Hu
6889746631e1218a0892818b
Xiaosong Zhang
6889746631e1218a0892818c
Linus
6889746631e1218a0892818d
Di Wang
6889746631e1218a0892818e
Jie Jiang
X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image
  Generative Models Great Again
Numerous efforts have been made to extend the ``next token prediction''
paradigm to visual contents, aiming to create a unified approach for both image
generation and understanding. Nevertheless, attempts to generate images through
autoregressive modeling with discrete tokens have been plagued by issues such
as low visual fidelity, distorted outputs, and failure to adhere to complex
instructions when rendering intricate details. These shortcomings are likely
attributed to cumulative errors during autoregressive inference or information
loss incurred during the discretization process. Probably due to this
challenge, recent research has increasingly shifted toward jointly training
image generation with diffusion objectives and language generation with
autoregressive objectives, moving away from unified modeling approaches. In
this work, we demonstrate that reinforcement learning can effectively mitigate
artifacts and largely enhance the generation quality of a discrete
autoregressive modeling method, thereby enabling seamless integration of image
and language generation. Our framework comprises a semantic image tokenizer, a
unified autoregressive model for both language and images, and an offline
diffusion decoder for image generation, termed X-Omni. X-Omni achieves
state-of-the-art performance in image generation tasks using a 7B language
model, producing images with high aesthetic quality while exhibiting strong
capabilities in following instructions and rendering long texts.
Reinforcement learning enhances discrete autoregressive modeling for image and language generation, achieving high-quality image generation and instruction-following capabilities using a unified framework.
autoregressive modeling discrete tokens visual fidelity distorted outputs cumulative errors information loss diffusion objectives language generation reinforcement learning semantic image tokenizer unified autoregressive model offline diffusion decoder X-Omni image generation aesthetic quality instruction-following