PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion
  Models
Personalized text-to-image generation has emerged as a powerful and
sought-after tool, empowering users to create customized images based on their
specific concepts and prompts. However, existing approaches to personalization
encounter multiple challenges, including long tuning times, large storage
requirements, the necessity for multiple input images per identity, and
limitations in preserving identity and editability. To address these obstacles,
we present PhotoVerse, an innovative methodology that incorporates a
dual-branch conditioning mechanism in both text and image domains, providing
effective control over the image generation process. Furthermore, we introduce
facial identity loss as a novel component to enhance the preservation of
identity during training. Remarkably, our proposed PhotoVerse eliminates the
need for test time tuning and relies solely on a single facial photo of the
target identity, significantly reducing the resource cost associated with image
generation. After a single training phase, our approach enables generating
high-quality images within only a few seconds. Moreover, our method can produce
diverse images that encompass various scenes and styles. The extensive
evaluation demonstrates the superior performance of our approach, which
achieves the dual objectives of preserving identity and facilitating
editability. Project page: https://photoverse2d.github.io/
650117784be99045d1c0bc5e
Li Chen
650117784be99045d1c0bc5f
Mengyi Zhao
admin_assigned
2023-09-13T07:36:06.355Z
6498038ece9190ebb8693034
/avatars/06ec2457932e05572d917ba286cdef25.svg
Zhao
user
Mengyi
650117784be99045d1c0bc60
Yiheng Liu
650117784be99045d1c0bc61
Mingxu Ding
admin_assigned
2023-09-13T07:36:33.134Z
643501b7901c5734bcb58b62
/avatars/983f34115768be49192a797e32efec3b.svg
MingxuDing
user
Dingmx
650117784be99045d1c0bc62
Yangyang Song
650117784be99045d1c0bc63
Shizun Wang
admin_assigned
2023-09-13T07:36:48.041Z
63492af03caab0136df08d25
https://cdn-avatars.huggingface.co/v1/production/uploads/63492af03caab0136df08d25/-aZzajKl38jT29A-kziCI.jpeg
Shizun Wang
user
littlepure2333
650117784be99045d1c0bc64
Xu Wang
650117784be99045d1c0bc65
Hao Yang
claimed_verified
2023-09-13T13:19:17.736Z
6322d55796d6c8518eaff8c5
/avatars/7309e77145674d01dbd842b0dcda171f.svg
Hao Yang
user
hayaalex
650117784be99045d1c0bc66
Jing Liu
650117784be99045d1c0bc67
Kang Du
650117784be99045d1c0bc68
Min Zheng
PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion
  Models
Personalized text-to-image generation has emerged as a powerful and
sought-after tool, empowering users to create customized images based on their
specific concepts and prompts. However, existing approaches to personalization
encounter multiple challenges, including long tuning times, large storage
requirements, the necessity for multiple input images per identity, and
limitations in preserving identity and editability. To address these obstacles,
we present PhotoVerse, an innovative methodology that incorporates a
dual-branch conditioning mechanism in both text and image domains, providing
effective control over the image generation process. Furthermore, we introduce
facial identity loss as a novel component to enhance the preservation of
identity during training. Remarkably, our proposed PhotoVerse eliminates the
need for test time tuning and relies solely on a single facial photo of the
target identity, significantly reducing the resource cost associated with image
generation. After a single training phase, our approach enables generating
high-quality images within only a few seconds. Moreover, our method can produce
diverse images that encompass various scenes and styles. The extensive
evaluation demonstrates the superior performance of our approach, which
achieves the dual objectives of preserving identity and facilitating
editability. Project page: https://photoverse2d.github.io/
PhotoVerse uses a dual-branch conditioning mechanism and facial identity loss to generate high-quality, identity-preserving images efficiently with minimal tuning.
duel-branch conditioning mechanism facial identity loss image generation identity preservation editability