LoRACLR: Contrastive Adaptation for Customization of Diffusion Models
Recent advances in text-to-image customization have enabled high-fidelity,
context-rich generation of personalized images, allowing specific concepts to
appear in a variety of scenarios. However, current methods struggle with
combining multiple personalized models, often leading to attribute entanglement
or requiring separate training to preserve concept distinctiveness. We present
LoRACLR, a novel approach for multi-concept image generation that merges
multiple LoRA models, each fine-tuned for a distinct concept, into a single,
unified model without additional individual fine-tuning. LoRACLR uses a
contrastive objective to align and merge the weight spaces of these models,
ensuring compatibility while minimizing interference. By enforcing distinct yet
cohesive representations for each concept, LoRACLR enables efficient, scalable
model composition for high-quality, multi-concept image synthesis. Our results
highlight the effectiveness of LoRACLR in accurately merging multiple concepts,
advancing the capabilities of personalized image generation.
675bd7f6a6ba99158dd02ad4
Enis Simsar
claimed_verified
2024-12-13T08:55:57.355Z
63412f2add8853dc7e306a4f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nqbteeQhS2dPA5LVy7dTD.png
Enis Simsar
user
enisimsar
675bd7f6a6ba99158dd02ad5
Thomas Hofmann
admin_assigned
2024-12-16T11:18:14.211Z
6630831ce888d89069e6276a
/avatars/b40d00ac8978405dd2ae66166ac969ba.svg
Thomas Hofmann
user
thofmann
675bd7f6a6ba99158dd02ad6
Federico Tombari
675bd7f6a6ba99158dd02ad7
Pinar Yanardag
claimed_verified
2025-06-02T07:54:55.506Z
683b9c7d4568ffd33e7334e1
/avatars/046bd5a78a24f3bc4e1f6cb6076970e0.svg
Pinar Yanardag
user
Pinguar
LoRACLR: Contrastive Adaptation for Customization of Diffusion Models
Recent advances in text-to-image customization have enabled high-fidelity,
context-rich generation of personalized images, allowing specific concepts to
appear in a variety of scenarios. However, current methods struggle with
combining multiple personalized models, often leading to attribute entanglement
or requiring separate training to preserve concept distinctiveness. We present
LoRACLR, a novel approach for multi-concept image generation that merges
multiple LoRA models, each fine-tuned for a distinct concept, into a single,
unified model without additional individual fine-tuning. LoRACLR uses a
contrastive objective to align and merge the weight spaces of these models,
ensuring compatibility while minimizing interference. By enforcing distinct yet
cohesive representations for each concept, LoRACLR enables efficient, scalable
model composition for high-quality, multi-concept image synthesis. Our results
highlight the effectiveness of LoRACLR in accurately merging multiple concepts,
advancing the capabilities of personalized image generation.
LoRACLR merges multiple LoRA models efficiently to generate high-quality images with distinct and cohesive multi-concept representations.
text-to-image customization high-fidelity context-rich personalized images LoRA models contrastive objective weight spaces model composition multi-concept image synthesis