Inserting Anybody in Diffusion Models via Celeb Basis
Exquisite demand exists for customizing the pretrained large text-to-image
model, e.g., Stable Diffusion, to generate innovative concepts, such
as the users themselves. However, the newly-added concept from previous
customization methods often shows weaker combination abilities than the
original ones even given several images during training. We thus propose a new
personalization method that allows for the seamless integration of a unique
individual into the pre-trained diffusion model using just one facial
photograph and only 1024 learnable parameters under 3
minutes. So as we can effortlessly generate stunning images of this person in
any pose or position, interacting with anyone and doing anything imaginable
from text prompts. To achieve this, we first analyze and build a well-defined
celeb basis from the embedding space of the pre-trained large text encoder.
Then, given one facial photo as the target identity, we generate its own
embedding by optimizing the weight of this basis and locking all other
parameters. Empowered by the proposed celeb basis, the new identity in our
customized model showcases a better concept combination ability than previous
personalization methods. Besides, our model can also learn several new
identities at once and interact with each other where the previous
customization model fails to. The code will be released.
64797cd85fa05d12dc35b988
Ge Yuan
64797cd85fa05d12dc35b989
Xiaodong Cun
admin_assigned
2023-06-02T08:05:39.087Z
63184c517ca1b876d99b7e0e
https://cdn-avatars.huggingface.co/v1/production/uploads/63184c517ca1b876d99b7e0e/b-qDExoeJuDXK0cJBZKnz.jpeg
Xiaodong Cun
user
vinthony
64797cd85fa05d12dc35b98a
Yong Zhang
64797cd85fa05d12dc35b98b
Maomao Li
64797cd85fa05d12dc35b98c
Chenyang Qi
admin_assigned
2023-06-02T08:06:56.908Z
642157a201c62c1e41fc526a
/avatars/73ae831a73947a04468b1e87d17da24d.svg
chenyangqi
user
chenyangqi
64797cd85fa05d12dc35b98d
Xintao Wang
admin_assigned
2023-06-02T08:07:19.530Z
60e272ca6c78a8c122b12127
https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
Xintao Wang
user
Xintao
64797cd85fa05d12dc35b98e
Ying Shan
64797cd85fa05d12dc35b98f
Huicheng Zheng
Inserting Anybody in Diffusion Models via Celeb Basis
Exquisite demand exists for customizing the pretrained large text-to-image
model, e.g., Stable Diffusion, to generate innovative concepts, such
as the users themselves. However, the newly-added concept from previous
customization methods often shows weaker combination abilities than the
original ones even given several images during training. We thus propose a new
personalization method that allows for the seamless integration of a unique
individual into the pre-trained diffusion model using just one facial
photograph and only 1024 learnable parameters under 3
minutes. So as we can effortlessly generate stunning images of this person in
any pose or position, interacting with anyone and doing anything imaginable
from text prompts. To achieve this, we first analyze and build a well-defined
celeb basis from the embedding space of the pre-trained large text encoder.
Then, given one facial photo as the target identity, we generate its own
embedding by optimizing the weight of this basis and locking all other
parameters. Empowered by the proposed celeb basis, the new identity in our
customized model showcases a better concept combination ability than previous
personalization methods. Besides, our model can also learn several new
identities at once and interact with each other where the previous
customization model fails to. The code will be released.
A novel method personalizes pre-trained diffusion models using minimal data and parameters, enabling high-quality image generation of new identities and interactions.
pretrained large text-to-image model Stable Diffusion diffusion model embedding space text encoder celeb basis image generation customization method concept combination ability identity interaction