UNCAGE: Contrastive Attention Guidance for Masked Generative
  Transformers in Text-to-Image Generation
Text-to-image (T2I) generation has been actively studied using Diffusion
Models and Autoregressive Models. Recently, Masked Generative Transformers have
gained attention as an alternative to Autoregressive Models to overcome the
inherent limitations of causal attention and autoregressive decoding through
bidirectional attention and parallel decoding, enabling efficient and
high-quality image generation. However, compositional T2I generation remains
challenging, as even state-of-the-art Diffusion Models often fail to accurately
bind attributes and achieve proper text-image alignment. While Diffusion Models
have been extensively studied for this issue, Masked Generative Transformers
exhibit similar limitations but have not been explored in this context. To
address this, we propose Unmasking with Contrastive Attention Guidance
(UNCAGE), a novel training-free method that improves compositional fidelity by
leveraging attention maps to prioritize the unmasking of tokens that clearly
represent individual objects. UNCAGE consistently improves performance in both
quantitative and qualitative evaluations across multiple benchmarks and
metrics, with negligible inference overhead. Our code is available at
https://github.com/furiosa-ai/uncage.
689bea5afab6fdd2e52ac7d1
Wonjun Kang
claimed_verified
2025-08-14T13:39:06.436Z
63e1d5247fbb6ae4d4f4cc8e
/avatars/8a8f700adf9e8000641c2c2f6bd56080.svg
Wonjun Kang
user
wjkang
689bea5afab6fdd2e52ac7d2
Byeongkeun Ahn
689bea5afab6fdd2e52ac7d3
Minjae Lee
claimed_verified
2025-08-26T09:50:53.077Z
6788cbfd6cdca10c9bb3dea5
/avatars/a0084b72d5b1a82d9c8a5aef1a230ffd.svg
Minjae Lee
user
FuriosaMJLEE
689bea5afab6fdd2e52ac7d4
Kevin Galim
689bea5afab6fdd2e52ac7d5
Seunghyuk Oh
claimed_verified
2025-08-13T07:13:14.929Z
64ae35dc00781825350e880b
https://cdn-avatars.huggingface.co/v1/production/uploads/64ae35dc00781825350e880b/VuON41yUDzNDACbvWAVhz.jpeg
Seunghyuk Oh
user
JakeOh
689bea5afab6fdd2e52ac7d6
Hyung Il Koo
689bea5afab6fdd2e52ac7d7
Nam Ik Cho
UNCAGE: Contrastive Attention Guidance for Masked Generative
  Transformers in Text-to-Image Generation
Text-to-image (T2I) generation has been actively studied using Diffusion
Models and Autoregressive Models. Recently, Masked Generative Transformers have
gained attention as an alternative to Autoregressive Models to overcome the
inherent limitations of causal attention and autoregressive decoding through
bidirectional attention and parallel decoding, enabling efficient and
high-quality image generation. However, compositional T2I generation remains
challenging, as even state-of-the-art Diffusion Models often fail to accurately
bind attributes and achieve proper text-image alignment. While Diffusion Models
have been extensively studied for this issue, Masked Generative Transformers
exhibit similar limitations but have not been explored in this context. To
address this, we propose Unmasking with Contrastive Attention Guidance
(UNCAGE), a novel training-free method that improves compositional fidelity by
leveraging attention maps to prioritize the unmasking of tokens that clearly
represent individual objects. UNCAGE consistently improves performance in both
quantitative and qualitative evaluations across multiple benchmarks and
metrics, with negligible inference overhead. Our code is available at
https://github.com/furiosa-ai/uncage.
UNCAGE, a training-free method using contrastive attention guidance, enhances compositional fidelity in text-to-image generation by prioritizing the unmasking of object-representing tokens.
Diffusion Models Autoregressive Models Masked Generative Transformers bidirectional attention parallel decoding compositional T2I generation attention maps text-image alignment UNCAGE