Training-Free Text-Guided Color Editing with Multi-Modal Diffusion
  Transformer
Text-guided color editing in images and videos is a fundamental yet unsolved
problem, requiring fine-grained manipulation of color attributes, including
albedo, light source color, and ambient lighting, while preserving physical
consistency in geometry, material properties, and light-matter interactions.
Existing training-free methods offer broad applicability across editing tasks
but struggle with precise color control and often introduce visual
inconsistency in both edited and non-edited regions. In this work, we present
ColorCtrl, a training-free color editing method that leverages the attention
mechanisms of modern Multi-Modal Diffusion Transformers (MM-DiT). By
disentangling structure and color through targeted manipulation of attention
maps and value tokens, our method enables accurate and consistent color
editing, along with word-level control of attribute intensity. Our method
modifies only the intended regions specified by the prompt, leaving unrelated
areas untouched. Extensive experiments on both SD3 and FLUX.1-dev demonstrate
that ColorCtrl outperforms existing training-free approaches and achieves
state-of-the-art performances in both edit quality and consistency.
Furthermore, our method surpasses strong commercial models such as FLUX.1
Kontext Max and GPT-4o Image Generation in terms of consistency. When extended
to video models like CogVideoX, our approach exhibits greater advantages,
particularly in maintaining temporal coherence and editing stability. Finally,
our method also generalizes to instruction-based editing diffusion models such
as Step1X-Edit and FLUX.1 Kontext dev, further demonstrating its versatility.
689ea8bea4caabb4320e5d9b
Zixin Yin
claimed_verified
2025-08-18T06:57:13.352Z
6503ccaf13d750b4604649e4
/avatars/96bebff9284d61f37c83e7da6a7e9bac.svg
Zixin Yin
user
zachary-yin
689ea8bea4caabb4320e5d9c
Xili Dai
689ea8bea4caabb4320e5d9d
Ling-Hao Chen
claimed_verified
2025-08-20T08:53:12.927Z
63109a4d61cab0446e48c83b
https://cdn-avatars.huggingface.co/v1/production/uploads/63109a4d61cab0446e48c83b/JQlVkQp0ok586ND1GmB0w.png
Ling-Hao Chen
user
EvanTHU
689ea8bea4caabb4320e5d9e
Deyu Zhou
689ea8bea4caabb4320e5d9f
Jianan Wang
689ea8bea4caabb4320e5da0
Duomin Wang
claimed_verified
2025-08-20T08:53:16.419Z
64ae9b88a22a179fc4d07992
/avatars/c9065f04a1188ea3129e56a90328ffd3.svg
wang
user
dorni
689ea8bea4caabb4320e5da1
Gang Yu
689ea8bea4caabb4320e5da2
Lionel M. Ni
689ea8bea4caabb4320e5da3
Lei Zhang
689ea8bea4caabb4320e5da4
Heung-Yeung Shum
Training-Free Text-Guided Color Editing with Multi-Modal Diffusion
  Transformer
Text-guided color editing in images and videos is a fundamental yet unsolved
problem, requiring fine-grained manipulation of color attributes, including
albedo, light source color, and ambient lighting, while preserving physical
consistency in geometry, material properties, and light-matter interactions.
Existing training-free methods offer broad applicability across editing tasks
but struggle with precise color control and often introduce visual
inconsistency in both edited and non-edited regions. In this work, we present
ColorCtrl, a training-free color editing method that leverages the attention
mechanisms of modern Multi-Modal Diffusion Transformers (MM-DiT). By
disentangling structure and color through targeted manipulation of attention
maps and value tokens, our method enables accurate and consistent color
editing, along with word-level control of attribute intensity. Our method
modifies only the intended regions specified by the prompt, leaving unrelated
areas untouched. Extensive experiments on both SD3 and FLUX.1-dev demonstrate
that ColorCtrl outperforms existing training-free approaches and achieves
state-of-the-art performances in both edit quality and consistency.
Furthermore, our method surpasses strong commercial models such as FLUX.1
Kontext Max and GPT-4o Image Generation in terms of consistency. When extended
to video models like CogVideoX, our approach exhibits greater advantages,
particularly in maintaining temporal coherence and editing stability. Finally,
our method also generalizes to instruction-based editing diffusion models such
as Step1X-Edit and FLUX.1 Kontext dev, further demonstrating its versatility.
ColorCtrl, a training-free method using Multi-Modal Diffusion Transformers, achieves precise and consistent color editing in images and videos with word-level control and superior performance compared to existing approaches.
Multi-Modal Diffusion Transformers MM-DiT attention mechanisms attention maps value tokens color editing edit quality consistency temporal coherence editing stability instruction-based editing diffusion models