Tuning-Free Image Editing with Fidelity and Editability via Unified
  Latent Diffusion Model
Balancing fidelity and editability is essential in text-based image editing
(TIE), where failures commonly lead to over- or under-editing issues. Existing
methods typically rely on attention injections for structure preservation and
leverage the inherent text alignment capabilities of pre-trained text-to-image
(T2I) models for editability, but they lack explicit and unified mechanisms to
properly balance these two objectives. In this work, we introduce UnifyEdit, a
tuning-free method that performs diffusion latent optimization to enable a
balanced integration of fidelity and editability within a unified framework.
Unlike direct attention injections, we develop two attention-based constraints:
a self-attention (SA) preservation constraint for structural fidelity, and a
cross-attention (CA) alignment constraint to enhance text alignment for
improved editability. However, simultaneously applying both constraints can
lead to gradient conflicts, where the dominance of one constraint results in
over- or under-editing. To address this challenge, we introduce an adaptive
time-step scheduler that dynamically adjusts the influence of these
constraints, guiding the diffusion latent toward an optimal balance. Extensive
quantitative and qualitative experiments validate the effectiveness of our
approach, demonstrating its superiority in achieving a robust balance between
structure preservation and text alignment across various editing tasks,
outperforming other state-of-the-art methods. The source code will be available
at https://github.com/CUC-MIPG/UnifyEdit.
67f5dc86015730c161ce291b
Qi Mao
claimed_verified
2025-04-10T06:45:41.166Z
6388a7e98a5dbe2f3dc61faa
https://cdn-avatars.huggingface.co/v1/production/uploads/6388a7e98a5dbe2f3dc61faa/Zj8pR7BG_wOQjsYzosb5p.jpeg
Qi Mao
user
HelenMao
67f5dc86015730c161ce291c
Lan Chen
67f5dc86015730c161ce291d
Yuchao Gu
admin_assigned
2025-04-09T16:43:17.644Z
63021630a35b21bd8a53305a
/avatars/7a7e8b39749eda61e57d8a1908726558.svg
Gu Yuchao
user
guyuchao
67f5dc86015730c161ce291e
Mike Zheng Shou
admin_assigned
2025-04-09T16:43:35.750Z
661ab3da2b14565c7acccf5c
/avatars/fa4fc03664803e02aede4d4c3d50b393.svg
Mike Zheng Shou
user
AnalMom
67f5dc86015730c161ce291f
Ming-Hsuan Yang
Tuning-Free Image Editing with Fidelity and Editability via Unified
  Latent Diffusion Model
Balancing fidelity and editability is essential in text-based image editing
(TIE), where failures commonly lead to over- or under-editing issues. Existing
methods typically rely on attention injections for structure preservation and
leverage the inherent text alignment capabilities of pre-trained text-to-image
(T2I) models for editability, but they lack explicit and unified mechanisms to
properly balance these two objectives. In this work, we introduce UnifyEdit, a
tuning-free method that performs diffusion latent optimization to enable a
balanced integration of fidelity and editability within a unified framework.
Unlike direct attention injections, we develop two attention-based constraints:
a self-attention (SA) preservation constraint for structural fidelity, and a
cross-attention (CA) alignment constraint to enhance text alignment for
improved editability. However, simultaneously applying both constraints can
lead to gradient conflicts, where the dominance of one constraint results in
over- or under-editing. To address this challenge, we introduce an adaptive
time-step scheduler that dynamically adjusts the influence of these
constraints, guiding the diffusion latent toward an optimal balance. Extensive
quantitative and qualitative experiments validate the effectiveness of our
approach, demonstrating its superiority in achieving a robust balance between
structure preservation and text alignment across various editing tasks,
outperforming other state-of-the-art methods. The source code will be available
at https://github.com/CUC-MIPG/UnifyEdit.
UnifyEdit uses diffusion latent optimization with adaptive constraints to balance structural fidelity and text alignment in text-based image editing.
diffusion latent optimization self-attention cross-attention adaptive time-step scheduler structure preservation text alignment text-based image editing T2I models over-editing under-editing