Single Image Iterative Subject-driven Generation and Editing
Personalizing image generation and editing is particularly challenging when
we only have a few images of the subject, or even a single image. A common
approach to personalization is concept learning, which can integrate the
subject into existing models relatively quickly, but produces images whose
quality tends to deteriorate quickly when the number of subject images is
small. Quality can be improved by pre-training an encoder, but training
restricts generation to the training distribution, and is time consuming. It is
still an open hard challenge to personalize image generation and editing from a
single image without training. Here, we present SISO, a novel, training-free
approach based on optimizing a similarity score with an input subject image.
More specifically, SISO iteratively generates images and optimizes the model
based on loss of similarity with the given subject image until a satisfactory
level of similarity is achieved, allowing plug-and-play optimization to any
image generator. We evaluated SISO in two tasks, image editing and image
generation, using a diverse data set of personal subjects, and demonstrate
significant improvements over existing methods in image quality, subject
fidelity, and background preservation.
67dd02594aa37abf77af416b
Yair Shpitzer
claimed_verified
2025-03-21T11:40:09.783Z
63eb8b1113a3eb9b0dc89d8c
/avatars/d9cb7bdf4f3d2218f7d84120a00054bb.svg
Yair Shpitzer
user
yairshp
67dd02594aa37abf77af416c
Gal Chechik
67dd02594aa37abf77af416d
Idan Schwartz
Single Image Iterative Subject-driven Generation and Editing
Personalizing image generation and editing is particularly challenging when
we only have a few images of the subject, or even a single image. A common
approach to personalization is concept learning, which can integrate the
subject into existing models relatively quickly, but produces images whose
quality tends to deteriorate quickly when the number of subject images is
small. Quality can be improved by pre-training an encoder, but training
restricts generation to the training distribution, and is time consuming. It is
still an open hard challenge to personalize image generation and editing from a
single image without training. Here, we present SISO, a novel, training-free
approach based on optimizing a similarity score with an input subject image.
More specifically, SISO iteratively generates images and optimizes the model
based on loss of similarity with the given subject image until a satisfactory
level of similarity is achieved, allowing plug-and-play optimization to any
image generator. We evaluated SISO in two tasks, image editing and image
generation, using a diverse data set of personal subjects, and demonstrate
significant improvements over existing methods in image quality, subject
fidelity, and background preservation.
SISO, a training-free method, enhances image generation and editing quality from single images by iteratively optimizing similarity between generated and subject images.
concept learning encoder iterative generation similarity score plug-and-play optimization image editing image quality subject fidelity background preservation