Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for
  Text-to-Image Generation
Recent works demonstrate that using reinforcement learning (RL) with quality
rewards can enhance the quality of generated images in text-to-image (T2I)
generation. However, a simple aggregation of multiple rewards may cause
over-optimization in certain metrics and degradation in others, and it is
challenging to manually find the optimal weights. An effective strategy to
jointly optimize multiple rewards in RL for T2I generation is highly desirable.
This paper introduces Parrot, a novel multi-reward RL framework for T2I
generation. Through the use of the batch-wise Pareto optimal selection, Parrot
automatically identifies the optimal trade-off among different rewards during
the RL optimization of the T2I generation. Additionally, Parrot employs a joint
optimization approach for the T2I model and the prompt expansion network,
facilitating the generation of quality-aware text prompts, thus further
enhancing the final image quality. To counteract the potential catastrophic
forgetting of the original user prompt due to prompt expansion, we introduce
original prompt centered guidance at inference time, ensuring that the
generated image remains faithful to the user input. Extensive experiments and a
user study demonstrate that Parrot outperforms several baseline methods across
various quality criteria, including aesthetics, human preference, image
sentiment, and text-image alignment.
65a0ad5990ea9d75eaa753b8
Seung Hyun Lee
admin_assigned
2024-01-12T09:35:17.506Z
6474b8c15aba8edfb2ef65aa
/avatars/3331316476ef9f79cd4294e28c88cfe0.svg
Seung Hyun Lee
user
Seanlee235
65a0ad5990ea9d75eaa753b9
Yinxiao Li
65a0ad5990ea9d75eaa753ba
Junjie Ke
admin_assigned
2024-01-12T09:36:21.063Z
64736b148b7a55cfa91e000f
/avatars/dcd2a666aba505575a79ffcc08a98c9d.svg
Ian Kerr
user
KeJunjie
65a0ad5990ea9d75eaa753bb
Innfarn Yoo
65a0ad5990ea9d75eaa753bc
Han Zhang
65a0ad5990ea9d75eaa753bd
Jiahui Yu
admin_assigned
2024-01-12T09:36:52.645Z
6174394801833e378dfc2517
/avatars/4989bcdda7ba1ae51cab8235c3e39275.svg
Jiahui Yu
user
jiahuiyu
65a0ad5990ea9d75eaa753be
Qifei Wang
65a0ad5990ea9d75eaa753bf
Fei Deng
65a0ad5990ea9d75eaa753c0
Glenn Entis
admin_assigned
2024-01-12T09:37:49.208Z
6317bf4033a4e2b5d3b0aaaa
/avatars/8a8b586e86d79520c5f0a866ed2f0b80.svg
Glenn Entis
user
GEinSF
65a0ad5990ea9d75eaa753c1
Junfeng He
admin_assigned
2024-01-12T09:37:55.773Z
658224bc083a345666a037bc
/avatars/a6719ec5f0a323b2504794dbf26a94d9.svg
Junfeng He
user
JF1980
65a0ad5990ea9d75eaa753c2
Gang Li
65a0ad5990ea9d75eaa753c3
Sangpil Kim
admin_assigned
2024-01-12T09:38:11.190Z
64fec50f6f77556d68d21fac
/avatars/89dc61984a1fc7f8a09624186fe9fd53.svg
sangpil
user
kimsangpil
65a0ad5990ea9d75eaa753c4
Irfan Essa
65a0ad5990ea9d75eaa753c5
Feng Yang
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for
  Text-to-Image Generation
Recent works demonstrate that using reinforcement learning (RL) with quality
rewards can enhance the quality of generated images in text-to-image (T2I)
generation. However, a simple aggregation of multiple rewards may cause
over-optimization in certain metrics and degradation in others, and it is
challenging to manually find the optimal weights. An effective strategy to
jointly optimize multiple rewards in RL for T2I generation is highly desirable.
This paper introduces Parrot, a novel multi-reward RL framework for T2I
generation. Through the use of the batch-wise Pareto optimal selection, Parrot
automatically identifies the optimal trade-off among different rewards during
the RL optimization of the T2I generation. Additionally, Parrot employs a joint
optimization approach for the T2I model and the prompt expansion network,
facilitating the generation of quality-aware text prompts, thus further
enhancing the final image quality. To counteract the potential catastrophic
forgetting of the original user prompt due to prompt expansion, we introduce
original prompt centered guidance at inference time, ensuring that the
generated image remains faithful to the user input. Extensive experiments and a
user study demonstrate that Parrot outperforms several baseline methods across
various quality criteria, including aesthetics, human preference, image
sentiment, and text-image alignment.
Parrot, a multi-reward RL framework, enhances text-to-image generation by automatically balancing multiple quality rewards and generating quality-aware text prompts without forgetting the original user input.
reinforcement learning quality rewards text-to-image generation batch-wise Pareto optimal selection prompt expansion network catastrophic forgetting original prompt centered guidance aesthetics human preference image sentiment text-image alignment