ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven
  Generation
This paper introduces a tuning-free method for both object insertion and
subject-driven generation. The task involves composing an object, given
multiple views, into a scene specified by either an image or text. Existing
methods struggle to fully meet the task's challenging objectives: (i)
seamlessly composing the object into the scene with photorealistic pose and
lighting, and (ii) preserving the object's identity. We hypothesize that
achieving these goals requires large scale supervision, but manually collecting
sufficient data is simply too expensive. The key observation in this paper is
that many mass-produced objects recur across multiple images of large unlabeled
datasets, in different scenes, poses, and lighting conditions. We use this
observation to create massive supervision by retrieving sets of diverse views
of the same object. This powerful paired dataset enables us to train a
straightforward text-to-image diffusion architecture to map the object and
scene descriptions to the composited image. We compare our method, ObjectMate,
with state-of-the-art methods for object insertion and subject-driven
generation, using a single or multiple references. Empirically, ObjectMate
achieves superior identity preservation and more photorealistic composition.
Differently from many other multi-reference methods, ObjectMate does not
require slow test-time tuning.
675b1648634bd80c4c1d84f8
Daniel Winter
admin_assigned
2024-12-16T10:16:31.261Z
675fccea42bc7570939a3642
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/x081h2YwSs9kIUgA5me8z.png
Daniel Winter
user
danielwinter
675b1648634bd80c4c1d84f9
Asaf Shul
675b1648634bd80c4c1d84fa
Matan Cohen
675b1648634bd80c4c1d84fb
Dana Berman
675b1648634bd80c4c1d84fc
Yael Pritch
675b1648634bd80c4c1d84fd
Alex Rav-Acha
675b1648634bd80c4c1d84fe
Yedid Hoshen
claimed_verified
2024-12-16T09:42:31.672Z
646cfc3b4220471ca0c56b20
/avatars/19d6ab141ec2cd25c1c3b45fd8f69910.svg
Yedid Hoshen
user
yedid
ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven
  Generation
This paper introduces a tuning-free method for both object insertion and
subject-driven generation. The task involves composing an object, given
multiple views, into a scene specified by either an image or text. Existing
methods struggle to fully meet the task's challenging objectives: (i)
seamlessly composing the object into the scene with photorealistic pose and
lighting, and (ii) preserving the object's identity. We hypothesize that
achieving these goals requires large scale supervision, but manually collecting
sufficient data is simply too expensive. The key observation in this paper is
that many mass-produced objects recur across multiple images of large unlabeled
datasets, in different scenes, poses, and lighting conditions. We use this
observation to create massive supervision by retrieving sets of diverse views
of the same object. This powerful paired dataset enables us to train a
straightforward text-to-image diffusion architecture to map the object and
scene descriptions to the composited image. We compare our method, ObjectMate,
with state-of-the-art methods for object insertion and subject-driven
generation, using a single or multiple references. Empirically, ObjectMate
achieves superior identity preservation and more photorealistic composition.
Differently from many other multi-reference methods, ObjectMate does not
require slow test-time tuning.
A straightforward text-to-image diffusion model trained on a massive dataset of diverse object views achieves superior identity preservation and photorealistic composition without requiring test-time tuning.
diffusion architecture text-to-image object insertion subject-driven generation