Interleaving Reasoning for Better Text-to-Image Generation
Unified multimodal understanding and generation models recently have achieve
significant improvement in image generation capability, yet a large gap remains
in instruction following and detail preservation compared to systems that
tightly couple comprehension with generation such as GPT-4o. Motivated by
recent advances in interleaving reasoning, we explore whether such reasoning
can further improve Text-to-Image (T2I) generation. We introduce Interleaving
Reasoning Generation (IRG), a framework that alternates between text-based
thinking and image synthesis: the model first produces a text-based thinking to
guide an initial image, then reflects on the result to refine fine-grained
details, visual quality, and aesthetics while preserving semantics. To train
IRG effectively, we propose Interleaving Reasoning Generation Learning (IRGL),
which targets two sub-goals: (1) strengthening the initial think-and-generate
stage to establish core content and base quality, and (2) enabling high-quality
textual reflection and faithful implementation of those refinements in a
subsequent image. We curate IRGL-300K, a dataset organized into six decomposed
learning modes that jointly cover learning text-based thinking, and full
thinking-image trajectories. Starting from a unified foundation model that
natively emits interleaved text-image outputs, our two-stage training first
builds robust thinking and reflection, then efficiently tunes the IRG pipeline
in the full thinking-image trajectory data. Extensive experiments show SoTA
performance, yielding absolute gains of 5-10 points on GenEval, WISE, TIIF,
GenAI-Bench, and OneIG-EN, alongside substantial improvements in visual quality
and fine-grained fidelity. The code, model weights and datasets will be
released in: https://github.com/Osilly/Interleaving-Reasoning-Generation .
68bfaf59207285de11b07cb4
Wenxuan Huang
claimed_verified
2025-09-09T13:47:14.259Z
67dc162ec8c00778e8689f42
https://cdn-avatars.huggingface.co/v1/production/uploads/67dc162ec8c00778e8689f42/_y_tO6W3ONOkOWbumAFXA.png
Wenxuan Huang
user
Osilly
68bfaf59207285de11b07cb5
Shuang Chen
68bfaf59207285de11b07cb6
Zheyong Xie
68bfaf59207285de11b07cb7
Shaosheng Cao
68bfaf59207285de11b07cb8
Shixiang Tang
68bfaf59207285de11b07cb9
Yufan Shen
68bfaf59207285de11b07cba
Qingyu Yin
68bfaf59207285de11b07cbb
Wenbo Hu
68bfaf59207285de11b07cbc
Xiaoman Wang
68bfaf59207285de11b07cbd
Yuntian Tang
68bfaf59207285de11b07cbe
Junbo Qiao
68bfaf59207285de11b07cbf
Yue Guo
68bfaf59207285de11b07cc0
Yao Hu
68bfaf59207285de11b07cc1
Zhenfei Yin
68bfaf59207285de11b07cc2
Philip Torr
68bfaf59207285de11b07cc3
Yu Cheng
68bfaf59207285de11b07cc4
Wanli Ouyang
68bfaf59207285de11b07cc5
Shaohui Lin
Interleaving Reasoning for Better Text-to-Image Generation
Unified multimodal understanding and generation models recently have achieve
significant improvement in image generation capability, yet a large gap remains
in instruction following and detail preservation compared to systems that
tightly couple comprehension with generation such as GPT-4o. Motivated by
recent advances in interleaving reasoning, we explore whether such reasoning
can further improve Text-to-Image (T2I) generation. We introduce Interleaving
Reasoning Generation (IRG), a framework that alternates between text-based
thinking and image synthesis: the model first produces a text-based thinking to
guide an initial image, then reflects on the result to refine fine-grained
details, visual quality, and aesthetics while preserving semantics. To train
IRG effectively, we propose Interleaving Reasoning Generation Learning (IRGL),
which targets two sub-goals: (1) strengthening the initial think-and-generate
stage to establish core content and base quality, and (2) enabling high-quality
textual reflection and faithful implementation of those refinements in a
subsequent image. We curate IRGL-300K, a dataset organized into six decomposed
learning modes that jointly cover learning text-based thinking, and full
thinking-image trajectories. Starting from a unified foundation model that
natively emits interleaved text-image outputs, our two-stage training first
builds robust thinking and reflection, then efficiently tunes the IRG pipeline
in the full thinking-image trajectory data. Extensive experiments show SoTA
performance, yielding absolute gains of 5-10 points on GenEval, WISE, TIIF,
GenAI-Bench, and OneIG-EN, alongside substantial improvements in visual quality
and fine-grained fidelity. The code, model weights and datasets will be
released in: https://github.com/Osilly/Interleaving-Reasoning-Generation .
Interleaving Reasoning Generation (IRG) framework alternates between text-based thinking and image synthesis to improve Text-to-Image generation, achieving state-of-the-art performance and enhanced visual quality.
Interleaving Reasoning Generation IRG Interleaving Reasoning Generation Learning IRGL text-based thinking image synthesis GenEval WISE TIIF GenAI-Bench OneIG-EN