PixelMan: Consistent Object Editing with Diffusion Models via Pixel
  Manipulation and Generation
Recent research explores the potential of Diffusion Models (DMs) for
consistent object editing, which aims to modify object position, size, and
composition, etc., while preserving the consistency of objects and background
without changing their texture and attributes. Current inference-time methods
often rely on DDIM inversion, which inherently compromises efficiency and the
achievable consistency of edited images. Recent methods also utilize energy
guidance which iteratively updates the predicted noise and can drive the
latents away from the original image, resulting in distortions. In this paper,
we propose PixelMan, an inversion-free and training-free method for achieving
consistent object editing via Pixel Manipulation and generation, where we
directly create a duplicate copy of the source object at target location in the
pixel space, and introduce an efficient sampling approach to iteratively
harmonize the manipulated object into the target location and inpaint its
original location, while ensuring image consistency by anchoring the edited
image to be generated to the pixel-manipulated image as well as by introducing
various consistency-preserving optimization techniques during inference.
Experimental evaluations based on benchmark datasets as well as extensive
visual comparisons show that in as few as 16 inference steps, PixelMan
outperforms a range of state-of-the-art training-based and training-free
methods (usually requiring 50 steps) on multiple consistent object editing
tasks.
6765152de07adde9c961fabb
Liyao Jiang
claimed_verified
2024-12-20T08:32:26.476Z
64ac49ccb7d86b40fd60a8dd
/avatars/e9f5482cffdd1d5917523a496a3805f0.svg
Liyao Jiang
user
LiyaoJiang
6765152de07adde9c961fabc
Negar Hassanpour
6765152de07adde9c961fabd
Mohammad Salameh
6765152de07adde9c961fabe
Mohammadreza Samadi
6765152de07adde9c961fabf
Jiao He
6765152de07adde9c961fac0
Fengyu Sun
6765152de07adde9c961fac1
Di Niu
PixelMan: Consistent Object Editing with Diffusion Models via Pixel
  Manipulation and Generation
Recent research explores the potential of Diffusion Models (DMs) for
consistent object editing, which aims to modify object position, size, and
composition, etc., while preserving the consistency of objects and background
without changing their texture and attributes. Current inference-time methods
often rely on DDIM inversion, which inherently compromises efficiency and the
achievable consistency of edited images. Recent methods also utilize energy
guidance which iteratively updates the predicted noise and can drive the
latents away from the original image, resulting in distortions. In this paper,
we propose PixelMan, an inversion-free and training-free method for achieving
consistent object editing via Pixel Manipulation and generation, where we
directly create a duplicate copy of the source object at target location in the
pixel space, and introduce an efficient sampling approach to iteratively
harmonize the manipulated object into the target location and inpaint its
original location, while ensuring image consistency by anchoring the edited
image to be generated to the pixel-manipulated image as well as by introducing
various consistency-preserving optimization techniques during inference.
Experimental evaluations based on benchmark datasets as well as extensive
visual comparisons show that in as few as 16 inference steps, PixelMan
outperforms a range of state-of-the-art training-based and training-free
methods (usually requiring 50 steps) on multiple consistent object editing
tasks.
PixelMan is an inversion-free and training-free method for consistent object editing that uses pixel manipulation and generation to efficiently edit object position, size, and composition while preserving consistency.
Diffusion Models DDIM inversion energy guidance pixel space pixel manipulation efficient sampling image consistency consistency-preserving optimization