StableIdentity: Inserting Anybody into Anywhere at First Sight
Recent advances in large pretrained text-to-image models have shown
unprecedented capabilities for high-quality human-centric generation, however,
customizing face identity is still an intractable problem. Existing methods
cannot ensure stable identity preservation and flexible editability, even with
several images for each subject during training. In this work, we propose
StableIdentity, which allows identity-consistent recontextualization with just
one face image. More specifically, we employ a face encoder with an identity
prior to encode the input face, and then land the face representation into a
space with an editable prior, which is constructed from celeb names. By
incorporating identity prior and editability prior, the learned identity can be
injected anywhere with various contexts. In addition, we design a masked
two-phase diffusion loss to boost the pixel-level perception of the input face
and maintain the diversity of generation. Extensive experiments demonstrate our
method outperforms previous customization methods. In addition, the learned
identity can be flexibly combined with the off-the-shelf modules such as
ControlNet. Notably, to the best knowledge, we are the first to directly inject
the identity learned from a single image into video/3D generation without
finetuning. We believe that the proposed StableIdentity is an important step to
unify image, video, and 3D customized generation models.
65b88335e0bde92c176880cc
Qinghe Wang
claimed_verified
2024-01-30T12:43:54.809Z
646f3418a6a58aa29505fd30
https://cdn-avatars.huggingface.co/v1/production/uploads/646f3418a6a58aa29505fd30/1z13rnpb6rsUgQsYumWPg.png
QINGHE WANG
user
DecoderWQH666
65b88335e0bde92c176880cd
Xu Jia
65b88335e0bde92c176880ce
Xiaomin Li
claimed_verified
2024-01-30T13:18:03.468Z
6423bfecadbaa37d50c3d9bc
/avatars/207f26c2d59d990f5740d620c3a399a5.svg
Xiaomin
user
XiaominDLUT
65b88335e0bde92c176880cf
Taiqing Li
claimed_verified
2024-01-30T12:43:52.319Z
64706f4122d12f52949b0117
/avatars/c6c7e89fbe1bbbf40e3871d5e4cfc3b4.svg
LITAIQING
user
LiTaiQing
65b88335e0bde92c176880d0
Liqian Ma
claimed_verified
2024-01-30T13:53:47.601Z
63075c0768bd6e30ddac805d
/avatars/7408b625508f31129f4e68fd38daeba3.svg
CharlieMa
user
CharlieMa
65b88335e0bde92c176880d1
Yunzhi Zhuge
65b88335e0bde92c176880d2
Huchuan Lu
StableIdentity: Inserting Anybody into Anywhere at First Sight
Recent advances in large pretrained text-to-image models have shown
unprecedented capabilities for high-quality human-centric generation, however,
customizing face identity is still an intractable problem. Existing methods
cannot ensure stable identity preservation and flexible editability, even with
several images for each subject during training. In this work, we propose
StableIdentity, which allows identity-consistent recontextualization with just
one face image. More specifically, we employ a face encoder with an identity
prior to encode the input face, and then land the face representation into a
space with an editable prior, which is constructed from celeb names. By
incorporating identity prior and editability prior, the learned identity can be
injected anywhere with various contexts. In addition, we design a masked
two-phase diffusion loss to boost the pixel-level perception of the input face
and maintain the diversity of generation. Extensive experiments demonstrate our
method outperforms previous customization methods. In addition, the learned
identity can be flexibly combined with the off-the-shelf modules such as
ControlNet. Notably, to the best knowledge, we are the first to directly inject
the identity learned from a single image into video/3D generation without
finetuning. We believe that the proposed StableIdentity is an important step to
unify image, video, and 3D customized generation models.
StableIdentity injects learned identity from a single face image into various contexts, including video and 3D, ensuring identity consistency and flexible edits.
face encoder identity prior editable prior masked two-phase diffusion loss ControlNet identity injection video generation 3D generation identity consistency