StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an
appropriate use of text prompts. However, ambiguities inherent in natural
language and out-of-distribution effects make it hard to synthesize image
styles, that leverage a specific design pattern, texture or material. In this
paper, we introduce StyleDrop, a method that enables the synthesis of images
that faithfully follow a specific style using a text-to-image model. The
proposed method is extremely versatile and captures nuances and details of a
user-provided style, such as color schemes, shading, design patterns, and local
and global effects. It efficiently learns a new style by fine-tuning very few
trainable parameters (less than 1% of total model parameters) and improving
the quality via iterative training with either human or automated feedback.
Better yet, StyleDrop is able to deliver impressive results even when the user
supplies only a single image that specifies the desired style. An extensive
study shows that, for the task of style tuning text-to-image models, StyleDrop
implemented on Muse convincingly outperforms other methods, including
DreamBooth and textual inversion on Imagen or Stable Diffusion. More results
are available at our project website: https://styledrop.github.io
64793ddf12ae749b62ed0b66
Kihyuk Sohn
64793ddf12ae749b62ed0b67
Nataniel Ruiz
claimed_verified
2023-06-05T10:07:35.037Z
6318c6e6cb116eab31d70a16
https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg
Nataniel
user
flavoredquark
64793ddf12ae749b62ed0b68
Kimin Lee
admin_assigned
2023-06-07T12:59:53.801Z
635097ec59bfa9a85d4207b2
/avatars/787085894e9e6538b6b3e3051efe9eea.svg
Kimin Lee
user
kiminle2
64793ddf12ae749b62ed0b69
Daniel Castro Chin
64793ddf12ae749b62ed0b6a
Irina Blok
64793ddf12ae749b62ed0b6b
Huiwen Chang
64793ddf12ae749b62ed0b6c
Jarred Barber
64793ddf12ae749b62ed0b6d
Lu Jiang
claimed_verified
2023-12-23T11:18:26.391Z
64a5cba3bea0116f8f7187a7
/avatars/97b3bad82e359c57d1ab7d6e3aa7748b.svg
Lu Jiang
user
roadjiang
64793ddf12ae749b62ed0b6e
Glenn Entis
admin_assigned
2023-06-07T13:01:14.885Z
6317bf4033a4e2b5d3b0aaaa
/avatars/8a8b586e86d79520c5f0a866ed2f0b80.svg
Glenn Entis
user
GEinSF
64793ddf12ae749b62ed0b6f
Yuanzhen Li
admin_assigned
2023-06-07T13:01:39.465Z
6334ee5831a2be3938c85639
/avatars/9821af26a585f54b595ec39f061f2250.svg
Yuanzhen Li
user
yzli
64793ddf12ae749b62ed0b70
Yuan Hao
64793ddf12ae749b62ed0b71
Irfan Essa
64793ddf12ae749b62ed0b72
Michael Rubinstein
64793ddf12ae749b62ed0b73
Dilip Krishnan
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an
appropriate use of text prompts. However, ambiguities inherent in natural
language and out-of-distribution effects make it hard to synthesize image
styles, that leverage a specific design pattern, texture or material. In this
paper, we introduce StyleDrop, a method that enables the synthesis of images
that faithfully follow a specific style using a text-to-image model. The
proposed method is extremely versatile and captures nuances and details of a
user-provided style, such as color schemes, shading, design patterns, and local
and global effects. It efficiently learns a new style by fine-tuning very few
trainable parameters (less than 1% of total model parameters) and improving
the quality via iterative training with either human or automated feedback.
Better yet, StyleDrop is able to deliver impressive results even when the user
supplies only a single image that specifies the desired style. An extensive
study shows that, for the task of style tuning text-to-image models, StyleDrop
implemented on Muse convincingly outperforms other methods, including
DreamBooth and textual inversion on Imagen or Stable Diffusion. More results
are available at our project website: https://styledrop.github.io
StyleDrop fine-tunes pre-trained text-to-image models to synthesize images in a specific style using minimal parameters and feedback.
text-to-image models StyleDrop fine-tuning parameter-efficient fine-tuning iterative training human feedback automated feedback Muse DreamBooth textual inversion Imagen Stable Diffusion