HOComp: Interaction-Aware Human-Object Composition
While existing image-guided composition methods may help insert a foreground
object onto a user-specified region of a background image, achieving natural
blending inside the region with the rest of the image unchanged, we observe
that these existing methods often struggle in synthesizing seamless
interaction-aware compositions when the task involves human-object
interactions. In this paper, we first propose HOComp, a novel approach for
compositing a foreground object onto a human-centric background image, while
ensuring harmonious interactions between the foreground object and the
background person and their consistent appearances. Our approach includes two
key designs: (1) MLLMs-driven Region-based Pose Guidance (MRPG), which utilizes
MLLMs to identify the interaction region as well as the interaction type (e.g.,
holding and lefting) to provide coarse-to-fine constraints to the generated
pose for the interaction while incorporating human pose landmarks to track
action variations and enforcing fine-grained pose constraints; and (2)
Detail-Consistent Appearance Preservation (DCAP), which unifies a shape-aware
attention modulation mechanism, a multi-view appearance loss, and a background
consistency loss to ensure consistent shapes/textures of the foreground and
faithful reproduction of the background human. We then propose the first
dataset, named Interaction-aware Human-Object Composition (IHOC), for the task.
Experimental results on our dataset show that HOComp effectively generates
harmonious human-object interactions with consistent appearances, and
outperforms relevant methods qualitatively and quantitatively.
68803aa000f4b5a05f2fbdb4
Dong Liang
68803aa000f4b5a05f2fbdb5
Jinyuan Jia
68803aa000f4b5a05f2fbdb6
Yuhao Liu
claimed_verified
2025-07-23T08:36:47.834Z
6351463b8445bbe32e944f6c
/avatars/ec0e8f378d5314d4af97d6c488771b3d.svg
Yuhao Liu
user
LeoLau
68803aa000f4b5a05f2fbdb7
Rynson W. H. Lau
HOComp: Interaction-Aware Human-Object Composition
While existing image-guided composition methods may help insert a foreground
object onto a user-specified region of a background image, achieving natural
blending inside the region with the rest of the image unchanged, we observe
that these existing methods often struggle in synthesizing seamless
interaction-aware compositions when the task involves human-object
interactions. In this paper, we first propose HOComp, a novel approach for
compositing a foreground object onto a human-centric background image, while
ensuring harmonious interactions between the foreground object and the
background person and their consistent appearances. Our approach includes two
key designs: (1) MLLMs-driven Region-based Pose Guidance (MRPG), which utilizes
MLLMs to identify the interaction region as well as the interaction type (e.g.,
holding and lefting) to provide coarse-to-fine constraints to the generated
pose for the interaction while incorporating human pose landmarks to track
action variations and enforcing fine-grained pose constraints; and (2)
Detail-Consistent Appearance Preservation (DCAP), which unifies a shape-aware
attention modulation mechanism, a multi-view appearance loss, and a background
consistency loss to ensure consistent shapes/textures of the foreground and
faithful reproduction of the background human. We then propose the first
dataset, named Interaction-aware Human-Object Composition (IHOC), for the task.
Experimental results on our dataset show that HOComp effectively generates
harmonious human-object interactions with consistent appearances, and
outperforms relevant methods qualitatively and quantitatively.
HOComp, a novel method using MLLMs and appearance preservation techniques, generates harmonious human-object interactions with consistent appearances in images.
MLLMs Region-based Pose Guidance Detail-Consistent Appearance Preservation shape-aware attention modulation multi-view appearance loss background consistency loss Interaction-aware Human-Object Composition dataset