Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance
  for Text-to-Image Generation
Despite the promising progress of recent autoregressive models in
text-to-image (T2I) generation, their ability to handle multi-attribute and
ambiguous prompts remains limited. To address these limitations, existing works
have applied chain-of-thought (CoT) to enable stage-aware visual synthesis and
employed reinforcement learning (RL) to improve reasoning capabilities.
However, most models provide reward signals only at the end of the generation
stage. This monolithic final-only guidance makes it difficult to identify which
stages contribute positively to the final outcome and may lead to suboptimal
policies. To tackle this issue, we propose a Visual-Chain of Guidance
(Visual-CoG) paradigm consisting of three stages: semantic reasoning, process
refining, and outcome evaluation, with stage-aware rewards providing immediate
guidance throughout the image generation pipeline. We further construct a
visual cognition benchmark, VisCog-Bench, which comprises four subtasks to
evaluate the effectiveness of semantic reasoning. Comprehensive evaluations on
GenEval, T2I-CompBench, and the proposed VisCog-Bench show improvements of 15%,
5%, and 19%, respectively, demonstrating the superior performance of the
proposed Visual-CoG. We will release all the resources soon.
68ad236486b21a0e2e358db3
Yaqi Li
68ad236486b21a0e2e358db4
Peng Chen
68ad236486b21a0e2e358db5
Mingyang Han
68ad236486b21a0e2e358db6
Bu Pi
68ad236486b21a0e2e358db7
Haoxiang Shi
68ad236486b21a0e2e358db8
Runzhou Zhao
68ad236486b21a0e2e358db9
Yang Yao
68ad236486b21a0e2e358dba
Xuan Zhang
68ad236486b21a0e2e358dbb
Jun Song
Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance
  for Text-to-Image Generation
Despite the promising progress of recent autoregressive models in
text-to-image (T2I) generation, their ability to handle multi-attribute and
ambiguous prompts remains limited. To address these limitations, existing works
have applied chain-of-thought (CoT) to enable stage-aware visual synthesis and
employed reinforcement learning (RL) to improve reasoning capabilities.
However, most models provide reward signals only at the end of the generation
stage. This monolithic final-only guidance makes it difficult to identify which
stages contribute positively to the final outcome and may lead to suboptimal
policies. To tackle this issue, we propose a Visual-Chain of Guidance
(Visual-CoG) paradigm consisting of three stages: semantic reasoning, process
refining, and outcome evaluation, with stage-aware rewards providing immediate
guidance throughout the image generation pipeline. We further construct a
visual cognition benchmark, VisCog-Bench, which comprises four subtasks to
evaluate the effectiveness of semantic reasoning. Comprehensive evaluations on
GenEval, T2I-CompBench, and the proposed VisCog-Bench show improvements of 15%,
5%, and 19%, respectively, demonstrating the superior performance of the
proposed Visual-CoG. We will release all the resources soon.
The Visual-Chain of Guidance (Visual-CoG) paradigm enhances text-to-image generation by providing stage-aware rewards, improving performance across multiple benchmarks.
autoregressive models text-to-image (T2I) generation chain-of-thought (CoT) reinforcement learning (RL) stage-aware visual synthesis semantic reasoning process refining outcome evaluation Visual-Chain of Guidance (Visual-CoG) VisCog-Bench GenEval T2I-CompBench