Rich Human Feedback for Text-to-Image Generation
Recent Text-to-Image (T2I) generation models such as Stable Diffusion and
Imagen have made significant progress in generating high-resolution images
based on text descriptions. However, many generated images still suffer from
issues such as artifacts/implausibility, misalignment with text descriptions,
and low aesthetic quality. Inspired by the success of Reinforcement Learning
with Human Feedback (RLHF) for large language models, prior works collected
human-provided scores as feedback on generated images and trained a reward
model to improve the T2I generation. In this paper, we enrich the feedback
signal by (i) marking image regions that are implausible or misaligned with the
text, and (ii) annotating which words in the text prompt are misrepresented or
missing on the image. We collect such rich human feedback on 18K generated
images and train a multimodal transformer to predict the rich feedback
automatically. We show that the predicted rich human feedback can be leveraged
to improve image generation, for example, by selecting high-quality training
data to finetune and improve the generative models, or by creating masks with
predicted heatmaps to inpaint the problematic regions. Notably, the
improvements generalize to models (Muse) beyond those used to generate the
images on which human feedback data were collected (Stable Diffusion variants).
6581296adefc9c0d300959bb
Youwei Liang
admin_assigned
2023-12-19T10:02:23.630Z
64a475aa33ded98516c6a37b
/avatars/3abf5f8c7f56cc8612ab590f1b75c1d5.svg
Youwei Liang
user
youweiliang
6581296adefc9c0d300959bc
Junfeng He
admin_assigned
2023-12-19T10:02:37.037Z
65794433ab3436f5adcb5649
/avatars/dde2ee2ba522bb1b657c9c782cac60b1.svg
hejunfeng
user
hejunfeng
6581296adefc9c0d300959bd
Gang Li
6581296adefc9c0d300959be
Peizhao Li
admin_assigned
2023-12-19T10:04:02.406Z
652442ce3a3b8c4b40d1dfac
/avatars/c6625570a7ac8c0233eb602652bbcbcb.svg
Peizhao Li
user
pzli
6581296adefc9c0d300959bf
Arseniy Klimovskiy
claimed_verified
2023-12-20T08:20:01.667Z
6581f9b82db4c45ef3e6449b
/avatars/12565c86e45f33feeef188955d329a86.svg
Arseniy Klimovskiy
user
aklimovsky
6581296adefc9c0d300959c0
Nicholas Carolan
claimed_verified
2023-12-19T19:21:43.761Z
6581e9972db4c45ef3e3aede
/avatars/244a0379a2c7bcfb6e70732971a50524.svg
Nicholas Carolan
user
ncarolan
6581296adefc9c0d300959c1
Jiao Sun
admin_assigned
2023-12-19T10:04:16.686Z
642cd2f7ad221e8f41d2f05e
/avatars/64ad33b67c10e3cb7b9b07744d108d8c.svg
Jiao Sun
user
sunjiao123sun
6581296adefc9c0d300959c2
Jordi Pont-Tuset
claimed_verified
2024-05-02T09:24:34.582Z
66335b4b07f864a2f94b403d
/avatars/0a290942c4a282b48f870500cc6424b5.svg
Jordi Pont Tuset
user
jponttuset
6581296adefc9c0d300959c3
Sarah Young
6581296adefc9c0d300959c4
Feng Yang
6581296adefc9c0d300959c5
Junjie Ke
admin_assigned
2023-12-19T10:05:49.856Z
64736b148b7a55cfa91e000f
/avatars/dcd2a666aba505575a79ffcc08a98c9d.svg
Ian Kerr
user
KeJunjie
6581296adefc9c0d300959c6
Krishnamurthy Dj Dvijotham
6581296adefc9c0d300959c7
Katie Collins
admin_assigned
2023-12-19T10:06:28.701Z
64514cd06db3be7ee57b6f34
/avatars/3276eeb344f57601e2a2199dd090935b.svg
Katie Collins
user
kmcollins
6581296adefc9c0d300959c8
Yiwen Luo
6581296adefc9c0d300959c9
Yang Li
6581296adefc9c0d300959ca
Kai J Kohlhoff
6581296adefc9c0d300959cb
Deepak Ramachandran
6581296adefc9c0d300959cc
Vidhya Navalpakkam
Rich Human Feedback for Text-to-Image Generation
Recent Text-to-Image (T2I) generation models such as Stable Diffusion and
Imagen have made significant progress in generating high-resolution images
based on text descriptions. However, many generated images still suffer from
issues such as artifacts/implausibility, misalignment with text descriptions,
and low aesthetic quality. Inspired by the success of Reinforcement Learning
with Human Feedback (RLHF) for large language models, prior works collected
human-provided scores as feedback on generated images and trained a reward
model to improve the T2I generation. In this paper, we enrich the feedback
signal by (i) marking image regions that are implausible or misaligned with the
text, and (ii) annotating which words in the text prompt are misrepresented or
missing on the image. We collect such rich human feedback on 18K generated
images and train a multimodal transformer to predict the rich feedback
automatically. We show that the predicted rich human feedback can be leveraged
to improve image generation, for example, by selecting high-quality training
data to finetune and improve the generative models, or by creating masks with
predicted heatmaps to inpaint the problematic regions. Notably, the
improvements generalize to models (Muse) beyond those used to generate the
images on which human feedback data were collected (Stable Diffusion variants).
Enriched human feedback is used to improve text-to-image generation through multimodal transformer training, enhancing quality and generalizability across different models.
Text-to-Image T2I generation Stable Diffusion Imagen artifacts misalignment aesthetic quality Reinforcement Learning with Human Feedback RLHF large language models reward model image regions text prompt multimodal transformer heatmap inpaint