GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
  for Visual Generation and Editing
Current image generation and editing methods primarily process textual
prompts as direct inputs without reasoning about visual composition and
explicit operations. We present Generation Chain-of-Thought (GoT), a novel
paradigm that enables generation and editing through an explicit language
reasoning process before outputting images. This approach transforms
conventional text-to-image generation and editing into a reasoning-guided
framework that analyzes semantic relationships and spatial arrangements. We
define the formulation of GoT and construct large-scale GoT datasets containing
over 9M samples with detailed reasoning chains capturing semantic-spatial
relationships. To leverage the advantages of GoT, we implement a unified
framework that integrates Qwen2.5-VL for reasoning chain generation with an
end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
Module. Experiments show our GoT framework achieves excellent performance on
both generation and editing tasks, with significant improvements over
baselines. Additionally, our approach enables interactive visual generation,
allowing users to explicitly modify reasoning steps for precise image
adjustments. GoT pioneers a new direction for reasoning-driven visual
generation and editing, producing images that better align with human intent.
To facilitate future research, we make our datasets, code, and pretrained
models publicly available at https://github.com/rongyaofang/GoT.
67d3a632db36a4d5d95dbcff
Rongyao Fang
claimed_verified
2025-03-14T08:56:14.110Z
65b8724123d948d884b379b1
/avatars/ce189d1d8d688c17912f9b869035b2d0.svg
Rongyao Fang
user
LucasFang
67d3a632db36a4d5d95dbd00
Chengqi Duan
admin_assigned
2025-03-14T09:07:01.612Z
64a2b496e2e19de17db7de65
/avatars/241448ca487833d6cc5d57bb1fdb6ee5.svg
Duan Chengqi
user
gogoduan
67d3a632db36a4d5d95dbd01
Kun Wang
67d3a632db36a4d5d95dbd02
Linjiang Huang
admin_assigned
2025-03-14T09:07:08.707Z
65fc7c824d36be78e66ba92d
/avatars/d4a55c820cae533f91724e062427516a.svg
Linjiang Huang
user
LjHuang
67d3a632db36a4d5d95dbd03
Hao Li
67d3a632db36a4d5d95dbd04
Shilin Yan
admin_assigned
2025-03-14T09:07:18.212Z
65273fea0ef49cfb783fa5c1
/avatars/0c9e204bc2151c8cc533311900d05a36.svg
shilinyan
user
shilinyan
67d3a632db36a4d5d95dbd05
Hao Tian
67d3a632db36a4d5d95dbd06
Xingyu Zeng
admin_assigned
2025-03-14T09:07:42.264Z
666d4a0fe70e5838d95aebee
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6dkjoFA_sOjCkjvcvozZ5.jpeg
zengxingyu
user
zengxingyu
67d3a632db36a4d5d95dbd07
Rui Zhao
67d3a632db36a4d5d95dbd08
Jifeng Dai
admin_assigned
2025-03-14T09:14:59.460Z
64686f7172d9180d4ac8b4e4
/avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
Jifeng Dai
user
daijifeng
67d3a632db36a4d5d95dbd09
Xihui Liu
admin_assigned
2025-03-14T09:14:50.924Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
67d3a632db36a4d5d95dbd0a
Hongsheng Li
admin_assigned
2025-03-14T09:14:43.402Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
  for Visual Generation and Editing
Current image generation and editing methods primarily process textual
prompts as direct inputs without reasoning about visual composition and
explicit operations. We present Generation Chain-of-Thought (GoT), a novel
paradigm that enables generation and editing through an explicit language
reasoning process before outputting images. This approach transforms
conventional text-to-image generation and editing into a reasoning-guided
framework that analyzes semantic relationships and spatial arrangements. We
define the formulation of GoT and construct large-scale GoT datasets containing
over 9M samples with detailed reasoning chains capturing semantic-spatial
relationships. To leverage the advantages of GoT, we implement a unified
framework that integrates Qwen2.5-VL for reasoning chain generation with an
end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
Module. Experiments show our GoT framework achieves excellent performance on
both generation and editing tasks, with significant improvements over
baselines. Additionally, our approach enables interactive visual generation,
allowing users to explicitly modify reasoning steps for precise image
adjustments. GoT pioneers a new direction for reasoning-driven visual
generation and editing, producing images that better align with human intent.
To facilitate future research, we make our datasets, code, and pretrained
models publicly available at https://github.com/rongyaofang/GoT.
A new reasoning-driven paradigm, GoT, improves text-to-image generation and editing through language understanding and reasoning chains, integrating Qwen2.5-VL and an enhanced diffusion model with a Semantic-Spatial Guidance Module.
Generation Chain-of-Thought GoT diffusion model Semantic-Spatial Guidance Module Qwen2.5-VL reasoning chain text-to-image generation interactive visual generation