RealCompo: Dynamic Equilibrium between Realism and Compositionality
  Improves Text-to-Image Diffusion Models
Diffusion models have achieved remarkable advancements in text-to-image
generation. However, existing models still have many difficulties when faced
with multiple-object compositional generation. In this paper, we propose a new
training-free and transferred-friendly text-to-image generation framework,
namely RealCompo, which aims to leverage the advantages of text-to-image and
layout-to-image models to enhance both realism and compositionality of the
generated images. An intuitive and novel balancer is proposed to dynamically
balance the strengths of the two models in denoising process, allowing
plug-and-play use of any model without extra training. Extensive experiments
show that our RealCompo consistently outperforms state-of-the-art text-to-image
models and layout-to-image models in multiple-object compositional generation
while keeping satisfactory realism and compositionality of the generated
images. Code is available at https://github.com/YangLing0818/RealCompo
65d56827543d28ba7d595852
Xinchen Zhang
claimed_verified
2024-07-29T07:10:13.151Z
653e5d31ffd60206c8b64bb5
https://cdn-avatars.huggingface.co/v1/production/uploads/653e5d31ffd60206c8b64bb5/bgztraPC27L6culMlJw4s.png
Xinchen Zhang
user
comin
65d56827543d28ba7d595853
Ling Yang
claimed_verified
2024-07-12T07:42:08.820Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
user
Lingaaaaaaa
65d56827543d28ba7d595854
Yaqi Cai
admin_assigned
2024-02-21T12:22:09.041Z
657fe0b1f010d76b6e709110
/avatars/9c08817484cfb5b238d6833d788fa0a3.svg
YaQi Cai
user
yaqicc
65d56827543d28ba7d595855
Zhaochen Yu
65d56827543d28ba7d595856
Jiake Xie
65d56827543d28ba7d595857
Ye Tian
claimed_verified
2025-03-25T08:23:52.054Z
64e357dd825f4133e7427bf8
https://cdn-avatars.huggingface.co/v1/production/uploads/64e357dd825f4133e7427bf8/HwaWhINrzkbXG6SHG2oyf.jpeg
tyfeld
user
tyfeld
65d56827543d28ba7d595858
Minkai Xu
claimed_verified
2024-03-05T20:57:45.302Z
64c0e950aa57599de1c75dad
/avatars/374d53317cbccc30fae70e5152ca13e0.svg
Minkai Xu
user
mkxu
65d56827543d28ba7d595859
Yong Tang
65d56827543d28ba7d59585a
Yujiu Yang
admin_assigned
2024-02-21T12:20:56.658Z
64ca1fe838837b12d5e529b7
/avatars/44a3ad9e59318784ac531993b5f69f6b.svg
Yujiu Yang
user
Thu-redrobot
65d56827543d28ba7d59585b
Bin Cui
RealCompo: Dynamic Equilibrium between Realism and Compositionality
  Improves Text-to-Image Diffusion Models
Diffusion models have achieved remarkable advancements in text-to-image
generation. However, existing models still have many difficulties when faced
with multiple-object compositional generation. In this paper, we propose a new
training-free and transferred-friendly text-to-image generation framework,
namely RealCompo, which aims to leverage the advantages of text-to-image and
layout-to-image models to enhance both realism and compositionality of the
generated images. An intuitive and novel balancer is proposed to dynamically
balance the strengths of the two models in denoising process, allowing
plug-and-play use of any model without extra training. Extensive experiments
show that our RealCompo consistently outperforms state-of-the-art text-to-image
models and layout-to-image models in multiple-object compositional generation
while keeping satisfactory realism and compositionality of the generated
images. Code is available at https://github.com/YangLing0818/RealCompo
RealCompo is a training-free text-to-image generation framework that enhances realism and compositionality by dynamically balancing text-to-image and layout-to-image models.
diffusion models text-to-image generation layout-to-image models multiple-object compositional generation balancer denoising process