MultiRef: Controllable Image Generation with Multiple Visual References
Visual designers naturally draw inspiration from multiple visual references,
combining diverse elements and aesthetic principles to create artwork. However,
current image generative frameworks predominantly rely on single-source inputs
-- either text prompts or individual reference images. In this paper, we focus
on the task of controllable image generation using multiple visual references.
We introduce MultiRef-bench, a rigorous evaluation framework comprising 990
synthetic and 1,000 real-world samples that require incorporating visual
content from multiple reference images. The synthetic samples are synthetically
generated through our data engine RefBlend, with 10 reference types and 33
reference combinations. Based on RefBlend, we further construct a dataset
MultiRef containing 38k high-quality images to facilitate further research. Our
experiments across three interleaved image-text models (i.e., OmniGen, ACE, and
Show-o) and six agentic frameworks (e.g., ChatDiT and LLM + SD) reveal that
even state-of-the-art systems struggle with multi-reference conditioning, with
the best model OmniGen achieving only 66.6% in synthetic samples and 79.0% in
real-world cases on average compared to the golden answer. These findings
provide valuable directions for developing more flexible and human-like
creative tools that can effectively integrate multiple sources of visual
inspiration. The dataset is publicly available at: https://multiref.github.io/.
689d51b2b083e610d741e9e4
Ruoxi Chen
689d51b2b083e610d741e9e5
Dongping Chen
689d51b2b083e610d741e9e6
Siyuan Wu
689d51b2b083e610d741e9e7
Sinan Wang
claimed_verified
2025-08-20T08:53:24.787Z
67b589ab0914159f2e33f7d2
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qZRV_ePJ1zaRxZrg7Ea4K.jpeg
Sinan Wang
user
wsnHowest
689d51b2b083e610d741e9e8
Shiyun Lang
689d51b2b083e610d741e9e9
Petr Sushko
689d51b2b083e610d741e9ea
Gaoyang Jiang
689d51b2b083e610d741e9eb
Yao Wan
689d51b2b083e610d741e9ec
Ranjay Krishna
MultiRef: Controllable Image Generation with Multiple Visual References
Visual designers naturally draw inspiration from multiple visual references,
combining diverse elements and aesthetic principles to create artwork. However,
current image generative frameworks predominantly rely on single-source inputs
-- either text prompts or individual reference images. In this paper, we focus
on the task of controllable image generation using multiple visual references.
We introduce MultiRef-bench, a rigorous evaluation framework comprising 990
synthetic and 1,000 real-world samples that require incorporating visual
content from multiple reference images. The synthetic samples are synthetically
generated through our data engine RefBlend, with 10 reference types and 33
reference combinations. Based on RefBlend, we further construct a dataset
MultiRef containing 38k high-quality images to facilitate further research. Our
experiments across three interleaved image-text models (i.e., OmniGen, ACE, and
Show-o) and six agentic frameworks (e.g., ChatDiT and LLM + SD) reveal that
even state-of-the-art systems struggle with multi-reference conditioning, with
the best model OmniGen achieving only 66.6% in synthetic samples and 79.0% in
real-world cases on average compared to the golden answer. These findings
provide valuable directions for developing more flexible and human-like
creative tools that can effectively integrate multiple sources of visual
inspiration. The dataset is publicly available at: https://multiref.github.io/.
Experiments with multiple image-text models and agentic frameworks show that even state-of-the-art systems struggle with generating images from multiple visual references, highlighting the need for more flexible creative tools.
MultiRef-bench RefBlend MultiRef OmniGen ACE Show-o ChatDiT LLM + SD