Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
We present Face0, a novel way to instantaneously condition a text-to-image
generation model on a face, in sample time, without any optimization procedures
such as fine-tuning or inversions. We augment a dataset of annotated images
with embeddings of the included faces and train an image generation model, on
the augmented dataset. Once trained, our system is practically identical at
inference time to the underlying base model, and is therefore able to generate
images, given a user-supplied face image and a prompt, in just a couple of
seconds. Our method achieves pleasing results, is remarkably simple, extremely
fast, and equips the underlying model with new capabilities, like controlling
the generated images both via text or via direct manipulation of the input face
embeddings. In addition, when using a fixed random vector instead of a face
embedding from a user supplied image, our method essentially solves the problem
of consistent character generation across images. Finally, while requiring
further research, we hope that our method, which decouples the model's textual
biases from its biases on faces, might be a step towards some mitigation of
biases in future text-to-image models.
6487d5275d164d8c7f8ed0bf
Dani Valevski
6487d5275d164d8c7f8ed0c0
Danny Wasserman
extracted_confirmed
2023-06-13T14:22:29.242Z
6488751b7e6b887ab77ce30d
/avatars/7fb9ef379f22d0190daab1e5ac548bc4.svg
Danny Lumen
user
DannyWasserman
6487d5275d164d8c7f8ed0c1
Yossi Matias
6487d5275d164d8c7f8ed0c2
Yaniv Leviathan
claimed_verified
2023-06-13T17:39:08.336Z
6394a2244db1a9bedd1f2229
/avatars/f04496811eb6873781e2ba606a2f97f0.svg
Yaniv Leviathan
user
yanivle
Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
We present Face0, a novel way to instantaneously condition a text-to-image
generation model on a face, in sample time, without any optimization procedures
such as fine-tuning or inversions. We augment a dataset of annotated images
with embeddings of the included faces and train an image generation model, on
the augmented dataset. Once trained, our system is practically identical at
inference time to the underlying base model, and is therefore able to generate
images, given a user-supplied face image and a prompt, in just a couple of
seconds. Our method achieves pleasing results, is remarkably simple, extremely
fast, and equips the underlying model with new capabilities, like controlling
the generated images both via text or via direct manipulation of the input face
embeddings. In addition, when using a fixed random vector instead of a face
embedding from a user supplied image, our method essentially solves the problem
of consistent character generation across images. Finally, while requiring
further research, we hope that our method, which decouples the model's textual
biases from its biases on faces, might be a step towards some mitigation of
biases in future text-to-image models.
A method to condition text-to-image generation models on face embeddings in real-time, enhancing generative capabilities and bias mitigation.
text-to-image generation face embeddings image generation model prompt inference time character generation biases