UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
This paper presents UltraEdit, a large-scale (approximately 4 million editing
samples), automatically generated dataset for instruction-based image editing.
Our key idea is to address the drawbacks in existing image editing datasets
like InstructPix2Pix and MagicBrush, and provide a systematic approach to
producing massive and high-quality image editing samples. UltraEdit offers
several distinct advantages: 1) It features a broader range of editing
instructions by leveraging the creativity of large language models (LLMs)
alongside in-context editing examples from human raters; 2) Its data sources
are based on real images, including photographs and artworks, which provide
greater diversity and reduced bias compared to datasets solely generated by
text-to-image models; 3) It also supports region-based editing, enhanced by
high-quality, automatically produced region annotations. Our experiments show
that canonical diffusion-based editing baselines trained on UltraEdit set new
records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
the crucial role of real image anchors and region-based editing data. The
dataset, code, and models can be found in https://ultra-editing.github.io.
668d5fd3ed63008dfaad9325
Haozhe Zhao
claimed_verified
2024-07-22T08:43:35.693Z
63ff09a02098b9ad105a09f6
/avatars/4409ca5d320050cf4c3df05962c7ff58.svg
Hans Zhao
user
BleachNick
668d5fd3ed63008dfaad9326
Xiaojian Ma
admin_assigned
2024-07-10T11:59:23.292Z
60dd0e36a15ddd7d2006d2e9
/avatars/8bd98177a79efbf295be8f6457683297.svg
Xiaojian Ma
user
jeasinema
668d5fd3ed63008dfaad9327
Liang Chen
admin_assigned
2024-07-10T11:59:31.061Z
658c481dd1c8b106727a8b73
/avatars/d34a7a62c3a524e5fdd2d5994348db58.svg
Liang Chen
user
liangchen-ms
668d5fd3ed63008dfaad9328
Shuzheng Si
claimed_verified
2024-10-11T07:45:34.954Z
637c99bbfe115289cfedfb44
https://cdn-avatars.huggingface.co/v1/production/uploads/637c99bbfe115289cfedfb44/344NN9KKF_XXTlVYaGaMW.png
ssz
user
ssz1111
668d5fd3ed63008dfaad9329
Rujie Wu
668d5fd3ed63008dfaad932a
Kaikai An
claimed_verified
2024-10-23T07:35:41.139Z
66c89152d33e34fbc29497d7
/avatars/bbddabf6532393951c4759e5915a065b.svg
KaikaiAn
user
kkk-an
668d5fd3ed63008dfaad932b
Peiyu Yu
668d5fd3ed63008dfaad932c
Minjia Zhang
admin_assigned
2024-07-10T12:00:06.212Z
6305b5e39d2531fabd195c5f
https://cdn-avatars.huggingface.co/v1/production/uploads/6305b5e39d2531fabd195c5f/oGS5leXVrB8Ct_BF96VkR.jpeg
Zhang
user
Minjia
668d5fd3ed63008dfaad932d
Qing Li
claimed_verified
2024-08-03T08:26:10.560Z
6455f8c2c8f569b995d603c9
/avatars/3cb2cad3ab123887c270234bb6a4ca43.svg
Qing Li
user
li-qing
668d5fd3ed63008dfaad932e
Baobao Chang
UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
This paper presents UltraEdit, a large-scale (approximately 4 million editing
samples), automatically generated dataset for instruction-based image editing.
Our key idea is to address the drawbacks in existing image editing datasets
like InstructPix2Pix and MagicBrush, and provide a systematic approach to
producing massive and high-quality image editing samples. UltraEdit offers
several distinct advantages: 1) It features a broader range of editing
instructions by leveraging the creativity of large language models (LLMs)
alongside in-context editing examples from human raters; 2) Its data sources
are based on real images, including photographs and artworks, which provide
greater diversity and reduced bias compared to datasets solely generated by
text-to-image models; 3) It also supports region-based editing, enhanced by
high-quality, automatically produced region annotations. Our experiments show
that canonical diffusion-based editing baselines trained on UltraEdit set new
records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
the crucial role of real image anchors and region-based editing data. The
dataset, code, and models can be found in https://ultra-editing.github.io.
UltraEdit, a large-scale dataset for instruction-based image editing, improves quality and diversity over existing datasets through real images and region annotations, enhancing diffusion-based editing methods.
UltraEdit large-scale dataset instruction-based image editing InstructPix2Pix MagicBrush large language models in-context editing examples real images photographs artworks region-based editing automatic region annotations canonical diffusion-based editing MagicBrush benchmark Emu-Edit benchmark