EditScore: Unlocking Online RL for Image Editing via High-Fidelity
  Reward Modeling
Instruction-guided image editing has achieved remarkable progress, yet
current models still face challenges with complex instructions and often
require multiple samples to produce a desired result. Reinforcement Learning
(RL) offers a promising solution, but its adoption in image editing has been
severely hindered by the lack of a high-fidelity, efficient reward signal. In
this work, we present a comprehensive methodology to overcome this barrier,
centered on the development of a state-of-the-art, specialized reward model. We
first introduce EditReward-Bench, a comprehensive benchmark to systematically
evaluate reward models on editing quality. Building on this benchmark, we
develop EditScore, a series of reward models (7B-72B) for evaluating the
quality of instruction-guided image editing. Through meticulous data curation
and filtering, EditScore effectively matches the performance of learning
proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy
tailored for the generative nature of EditScore, our largest variant even
surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity
reward model is the key to unlocking online RL for image editing. Our
experiments show that, while even the largest open-source VLMs fail to provide
an effective learning signal, EditScore enables efficient and robust policy
optimization. Applying our framework to a strong base model, OmniGen2, results
in a final model that shows a substantial and consistent performance uplift.
Overall, this work provides the first systematic path from benchmarking to
reward modeling to RL training in image editing, showing that a high-fidelity,
domain-specialized reward model is the key to unlocking the full potential of
RL in this domain.
68db3dd7d2bf1f4b15ec72c4
Xin Luo
claimed_verified
2025-10-05T12:47:50.465Z
641bd1737c21ab946bf69aff
/avatars/83759075ad893a69a0c2cf5493d7e988.svg
xin luo
user
sienna223
68db3dd7d2bf1f4b15ec72c5
Jiahao Wang
68db3dd7d2bf1f4b15ec72c6
Chenyuan Wu
claimed_verified
2025-10-11T14:06:01.564Z
65f19fa7f591e4538b65dea5
/avatars/a38e65701e1d2eb3eb93335d6d0b937c.svg
Chenyuan Wu
user
wcyno23
68db3dd7d2bf1f4b15ec72c7
Shitao Xiao
68db3dd7d2bf1f4b15ec72c8
Xiyan Jiang
68db3dd7d2bf1f4b15ec72c9
Defu Lian
68db3dd7d2bf1f4b15ec72ca
Jiajun Zhang
68db3dd7d2bf1f4b15ec72cb
Dong Liu
68db3dd7d2bf1f4b15ec72cc
Zheng liu
EditScore: Unlocking Online RL for Image Editing via High-Fidelity
  Reward Modeling
Instruction-guided image editing has achieved remarkable progress, yet
current models still face challenges with complex instructions and often
require multiple samples to produce a desired result. Reinforcement Learning
(RL) offers a promising solution, but its adoption in image editing has been
severely hindered by the lack of a high-fidelity, efficient reward signal. In
this work, we present a comprehensive methodology to overcome this barrier,
centered on the development of a state-of-the-art, specialized reward model. We
first introduce EditReward-Bench, a comprehensive benchmark to systematically
evaluate reward models on editing quality. Building on this benchmark, we
develop EditScore, a series of reward models (7B-72B) for evaluating the
quality of instruction-guided image editing. Through meticulous data curation
and filtering, EditScore effectively matches the performance of learning
proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy
tailored for the generative nature of EditScore, our largest variant even
surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity
reward model is the key to unlocking online RL for image editing. Our
experiments show that, while even the largest open-source VLMs fail to provide
an effective learning signal, EditScore enables efficient and robust policy
optimization. Applying our framework to a strong base model, OmniGen2, results
in a final model that shows a substantial and consistent performance uplift.
Overall, this work provides the first systematic path from benchmarking to
reward modeling to RL training in image editing, showing that a high-fidelity,
domain-specialized reward model is the key to unlocking the full potential of
RL in this domain.
A specialized reward model, EditScore, enables effective reinforcement learning for instruction-guided image editing by providing a high-fidelity reward signal.
Reinforcement Learning reward model EditReward-Bench EditScore OmniGen2 policy optimization