Imagine yourself: Tuning-Free Personalized Image Generation
Diffusion models have demonstrated remarkable efficacy across various
image-to-image tasks. In this research, we introduce Imagine yourself, a
state-of-the-art model designed for personalized image generation. Unlike
conventional tuning-based personalization techniques, Imagine yourself operates
as a tuning-free model, enabling all users to leverage a shared framework
without individualized adjustments. Moreover, previous work met challenges
balancing identity preservation, following complex prompts and preserving good
visual quality, resulting in models having strong copy-paste effect of the
reference images. Thus, they can hardly generate images following prompts that
require significant changes to the reference image, \eg, changing facial
expression, head and body poses, and the diversity of the generated images is
low. To address these limitations, our proposed method introduces 1) a new
synthetic paired data generation mechanism to encourage image diversity, 2) a
fully parallel attention architecture with three text encoders and a fully
trainable vision encoder to improve the text faithfulness, and 3) a novel
coarse-to-fine multi-stage finetuning methodology that gradually pushes the
boundary of visual quality. Our study demonstrates that Imagine yourself
surpasses the state-of-the-art personalization model, exhibiting superior
capabilities in identity preservation, visual quality, and text alignment. This
model establishes a robust foundation for various personalization applications.
Human evaluation results validate the model's SOTA superiority across all
aspects (identity preservation, text faithfulness, and visual appeal) compared
to the previous personalization models.
66f0ecb3a4c40b65f651c498
Zecheng He
admin_assigned
2024-09-23T07:13:18.477Z
641b7d42ec5b871c0bcf2955
/avatars/f6e24742b4befa889e72c7e5eefd6cf6.svg
He
user
zechengh
66f0ecb3a4c40b65f651c499
Bo Sun
claimed_verified
2024-09-25T07:34:03.233Z
6552a36bd04b38b354592bd5
https://cdn-avatars.huggingface.co/v1/production/uploads/6552a36bd04b38b354592bd5/KI8qpshurUOAkri-MKsm_.jpeg
Bo Sun
user
bsun0802
66f0ecb3a4c40b65f651c49a
Felix Juefei-Xu
admin_assigned
2024-09-23T07:22:46.770Z
6444e8911cfc9ae6bb3ad216
/avatars/8c06e064cf24789e4131f7af06dac86b.svg
Xu
user
FelixXu
66f0ecb3a4c40b65f651c49b
Haoyu Ma
admin_assigned
2024-09-23T07:22:27.813Z
650a8979c19e5b4c8a6ff062
https://cdn-avatars.huggingface.co/v1/production/uploads/650a8979c19e5b4c8a6ff062/64_JuECX_k_-uK7m7nlua.jpeg
Haoyu Ma
user
haoyum1997
66f0ecb3a4c40b65f651c49c
Ankit Ramchandani
admin_assigned
2024-09-23T07:22:11.834Z
667c9d6aaeba4a9f637f4a93
/avatars/3167b482986b822a923966a459f3e362.svg
Ankit Ramchandani
user
ankit61
66f0ecb3a4c40b65f651c49d
Vincent Cheung
claimed_verified
2024-09-24T07:53:09.585Z
6351060957597a118689f9f9
/avatars/dde4af7d33d15e1bca62ad549c2b8649.svg
Vincent Cheung
user
veenix
66f0ecb3a4c40b65f651c49e
Siddharth Shah
claimed_verified
2024-09-24T07:53:11.787Z
66f1a12a9feec2e239601723
/avatars/35d2895042c34387f5e23b545f8fb540.svg
Siddharth Shah
user
siddshah
66f0ecb3a4c40b65f651c49f
Anmol Kalia
admin_assigned
2024-09-23T07:21:24.727Z
6573c2a8769f3ee9bdf98a74
/avatars/df40b6153ec7368561049746a9c8adc2.svg
Anmol Kalia
user
anmolkalia
66f0ecb3a4c40b65f651c4a0
Harihar Subramanyam
admin_assigned
2024-09-23T07:21:18.664Z
6303da377373aacccd868d0f
/avatars/88813105f863a0b2125fbe8bfffdec0e.svg
Harihar Subramanyam
user
Harihar
66f0ecb3a4c40b65f651c4a1
Alireza Zareian
66f0ecb3a4c40b65f651c4a2
Li Chen
66f0ecb3a4c40b65f651c4a3
Ankit Jain
66f0ecb3a4c40b65f651c4a4
Ning Zhang
66f0ecb3a4c40b65f651c4a5
Peizhao Zhang
admin_assigned
2024-09-23T07:20:22.954Z
6433a635a5aed21dd110381e
/avatars/b1fa4fe1c8bd7989875233820fa425af.svg
Peizhao Zhang
user
stzpz
66f0ecb3a4c40b65f651c4a6
Roshan Sumbaly
admin_assigned
2024-09-23T07:20:16.872Z
6515c595f464644a73b50064
/avatars/119102c41fb8a7986c72b3f55ec6200f.svg
Roshan sumbaly
user
rsumbaly
66f0ecb3a4c40b65f651c4a7
Peter Vajda
admin_assigned
2024-09-23T07:20:10.159Z
636f4b35af79dd193603342b
/avatars/577328195738d92ac41a3d0ff3108d7c.svg
Peter Vajda
user
vaphab
66f0ecb3a4c40b65f651c4a8
Animesh Sinha
admin_assigned
2024-09-23T07:19:52.007Z
63772d28e069bdcdbb8e4a29
/avatars/be71b00e009f2093a3ff79f07815e68e.svg
Animesh Sinha
user
animeshsinha
Imagine yourself: Tuning-Free Personalized Image Generation
Diffusion models have demonstrated remarkable efficacy across various
image-to-image tasks. In this research, we introduce Imagine yourself, a
state-of-the-art model designed for personalized image generation. Unlike
conventional tuning-based personalization techniques, Imagine yourself operates
as a tuning-free model, enabling all users to leverage a shared framework
without individualized adjustments. Moreover, previous work met challenges
balancing identity preservation, following complex prompts and preserving good
visual quality, resulting in models having strong copy-paste effect of the
reference images. Thus, they can hardly generate images following prompts that
require significant changes to the reference image, \eg, changing facial
expression, head and body poses, and the diversity of the generated images is
low. To address these limitations, our proposed method introduces 1) a new
synthetic paired data generation mechanism to encourage image diversity, 2) a
fully parallel attention architecture with three text encoders and a fully
trainable vision encoder to improve the text faithfulness, and 3) a novel
coarse-to-fine multi-stage finetuning methodology that gradually pushes the
boundary of visual quality. Our study demonstrates that Imagine yourself
surpasses the state-of-the-art personalization model, exhibiting superior
capabilities in identity preservation, visual quality, and text alignment. This
model establishes a robust foundation for various personalization applications.
Human evaluation results validate the model's SOTA superiority across all
aspects (identity preservation, text faithfulness, and visual appeal) compared
to the previous personalization models.
Imagine yourself is a tuning-free diffusion model for personalized image generation that enhances identity preservation, text alignment, and visual quality through synthetic data generation, parallel attention architecture, and multi-stage fine-tuning.
diffusion models personalization synthetic paired data parallel attention architecture text encoders vision encoder multi-stage fine-tuning identity preservation text alignment visual quality