TokenVerse: Versatile Multi-concept Personalization in Token Modulation
  Space
We present TokenVerse -- a method for multi-concept personalization,
leveraging a pre-trained text-to-image diffusion model. Our framework can
disentangle complex visual elements and attributes from as little as a single
image, while enabling seamless plug-and-play generation of combinations of
concepts extracted from multiple images. As opposed to existing works,
TokenVerse can handle multiple images with multiple concepts each, and supports
a wide-range of concepts, including objects, accessories, materials, pose, and
lighting. Our work exploits a DiT-based text-to-image model, in which the input
text affects the generation through both attention and modulation (shift and
scale). We observe that the modulation space is semantic and enables localized
control over complex concepts. Building on this insight, we devise an
optimization-based framework that takes as input an image and a text
description, and finds for each word a distinct direction in the modulation
space. These directions can then be used to generate new images that combine
the learned concepts in a desired configuration. We demonstrate the
effectiveness of TokenVerse in challenging personalization settings, and
showcase its advantages over existing methods. project's webpage in
https://token-verse.github.io/
6790db6588d8a90790d99ed4
Daniel Garibi
admin_assigned
2025-01-22T15:32:53.662Z
65fbf768980d4143b0d3ab52
/avatars/018a26ce944860e9ab97ca43765e7d76.svg
Daniel Garibi
user
garibida
6790db6588d8a90790d99ed5
Shahar Yadin
admin_assigned
2025-01-22T15:33:02.856Z
66ab799e5e4f9699e476856e
/avatars/942943ed751c9703c34782fd7bd92be2.svg
Shahar Yadin
user
shaharyadin
6790db6588d8a90790d99ed6
Roni Paiss
admin_assigned
2025-01-22T15:33:11.237Z
62cab8aeb189b6164d26bfbe
/avatars/7ed117e375dd9280a5750a242cae24ca.svg
Roni Paiss
user
RoniPaiss
6790db6588d8a90790d99ed7
Omer Tov
admin_assigned
2025-01-22T15:33:26.476Z
623889093657dc674c30954e
/avatars/a5a193d231e01448cc78ed5bba52bff5.svg
Omer Tov
user
omertov
6790db6588d8a90790d99ed8
Shiran Zada
admin_assigned
2025-01-22T15:33:37.040Z
63510f62cba4ff2e81cb0492
/avatars/2375a841491de8f40f66b6d0fb0df7b1.svg
Shiran Zada
user
shiranzada
6790db6588d8a90790d99ed9
Ariel Ephrat
admin_assigned
2025-01-22T15:33:44.386Z
65b6ada5ff5235e6ca9551c9
/avatars/2c558fd133bc1d0cd1b5d7f39d1968ee.svg
Ariel Ephrat
user
arielephrat
6790db6588d8a90790d99eda
Tomer Michaeli
admin_assigned
2025-01-22T15:33:58.984Z
6457c3f50664b05a5b1d941d
/avatars/0132be4d068d4b314f6b498b65411461.svg
Tomer Michaeli
user
Michato
6790db6588d8a90790d99edb
Inbar Mosseri
admin_assigned
2025-01-22T15:34:07.262Z
670f75a77bb118f080d29eef
/avatars/134687f97a476be6db5133e26de83ca5.svg
Inbar Mosseri
user
inbarm
6790db6588d8a90790d99edc
Tali Dekel
admin_assigned
2025-01-22T15:34:15.392Z
631cddec68f7da9ad24f6fc7
/avatars/7d4f1ce805e5889ca6594bd4a93f2583.svg
Tali Dekel
user
talidekel
TokenVerse: Versatile Multi-concept Personalization in Token Modulation
  Space
We present TokenVerse -- a method for multi-concept personalization,
leveraging a pre-trained text-to-image diffusion model. Our framework can
disentangle complex visual elements and attributes from as little as a single
image, while enabling seamless plug-and-play generation of combinations of
concepts extracted from multiple images. As opposed to existing works,
TokenVerse can handle multiple images with multiple concepts each, and supports
a wide-range of concepts, including objects, accessories, materials, pose, and
lighting. Our work exploits a DiT-based text-to-image model, in which the input
text affects the generation through both attention and modulation (shift and
scale). We observe that the modulation space is semantic and enables localized
control over complex concepts. Building on this insight, we devise an
optimization-based framework that takes as input an image and a text
description, and finds for each word a distinct direction in the modulation
space. These directions can then be used to generate new images that combine
the learned concepts in a desired configuration. We demonstrate the
effectiveness of TokenVerse in challenging personalization settings, and
showcase its advantages over existing methods. project's webpage in
https://token-verse.github.io/
TokenVerse uses a DiT-based text-to-image model to disentangle and personalize complex visual concepts from multiple images using an optimization-based framework.
DiT-based text-to-image model attention modulation modulation space optimization-based framework