aMUSEd: An Open MUSE Reproduction
We present aMUSEd, an open-source, lightweight masked image model (MIM) for
text-to-image generation based on MUSE. With 10 percent of MUSE's parameters,
aMUSEd is focused on fast image generation. We believe MIM is under-explored
compared to latent diffusion, the prevailing approach for text-to-image
generation. Compared to latent diffusion, MIM requires fewer inference steps
and is more interpretable. Additionally, MIM can be fine-tuned to learn
additional styles with only a single image. We hope to encourage further
exploration of MIM by demonstrating its effectiveness on large-scale
text-to-image generation and releasing reproducible training code. We also
release checkpoints for two models which directly produce images at 256x256 and
512x512 resolutions.
65961d6012c3ee130fac1d2d
Suraj Patil
claimed_verified
2024-01-04T13:59:49.014Z
5ec0135ded25d76864d553f1
https://cdn-avatars.huggingface.co/v1/production/uploads/1600014820272-5ec0135ded25d76864d553f1.jpeg
Suraj Patil
user
valhalla
65961d6012c3ee130fac1d2e
William Berman
65961d6012c3ee130fac1d2f
Robin Rombach
65961d6012c3ee130fac1d30
Patrick von Platen
claimed_verified
2024-01-04T14:23:40.680Z
5dfcb1aada6d0311fd3d5448
https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg
Patrick von Platen
user
patrickvonplaten
aMUSEd: An Open MUSE Reproduction
We present aMUSEd, an open-source, lightweight masked image model (MIM) for
text-to-image generation based on MUSE. With 10 percent of MUSE's parameters,
aMUSEd is focused on fast image generation. We believe MIM is under-explored
compared to latent diffusion, the prevailing approach for text-to-image
generation. Compared to latent diffusion, MIM requires fewer inference steps
and is more interpretable. Additionally, MIM can be fine-tuned to learn
additional styles with only a single image. We hope to encourage further
exploration of MIM by demonstrating its effectiveness on large-scale
text-to-image generation and releasing reproducible training code. We also
release checkpoints for two models which directly produce images at 256x256 and
512x512 resolutions.
aMUSEd, a masked image model with 10% of MUSE's parameters, generates images quickly and efficiently, requiring fewer inference steps and offering easier fine-tuning compared to latent diffusion.
masked image model latent diffusion inference steps fine-tuning