BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion
Textual Inversion remains a popular method for personalizing diffusion
models, in order to teach models new subjects and styles. We note that textual
inversion has been underexplored using alternatives to the UNet, and experiment
with textual inversion with a vision transformer. We also seek to optimize
textual inversion using a strategy that does not require explicit use of the
UNet and its idiosyncratic layers, so we add bonus tokens and enforce
orthogonality. We find the use of the bonus token improves adherence to the
source images and the use of the vision transformer improves adherence to the
prompt. Code is available at https://github.com/jamesBaker361/tex_inv_plus.
66b9cb73046cedd83757190f
James Baker
extracted_pending
2024-08-12T08:44:37.400Z
637317efbd81fae2b3aa400c
/avatars/898fa184c2d2fb7479ed097b4ddee5e9.svg
James Baker
user
jlbaker361
BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion
Textual Inversion remains a popular method for personalizing diffusion
models, in order to teach models new subjects and styles. We note that textual
inversion has been underexplored using alternatives to the UNet, and experiment
with textual inversion with a vision transformer. We also seek to optimize
textual inversion using a strategy that does not require explicit use of the
UNet and its idiosyncratic layers, so we add bonus tokens and enforce
orthogonality. We find the use of the bonus token improves adherence to the
source images and the use of the vision transformer improves adherence to the
prompt. Code is available at https://github.com/jamesBaker361/tex_inv_plus.
Textual inversion is optimized using vision transformers and bonus tokens to improve adherence to source images and prompts without explicit use of UNet.
textual inversion UNet vision transformer bonus tokens orthogonality