Learning Human-Object Interaction as Groups
Jiajun Hong, Jianan Wei, Wenguan Wang
Human-Object Interaction (HOI) detection aims to localize human-object pairs and identify interactive relationships between them. To aggregate the contextual cues of the scene, current HOI methods either propagate information across all the detected entities via self-attention mechanisms, or establish message passing between humans and objects using a bipartite graph architecture. However, theyall neglect inherent social attributes within human-centric scenarios, which are collective and beyond pairwise. In light of this, we revisit the relation modeling in HOI from a group view, and propose GroupHOI, a novel framework that propagates contextual information in terms of geometric proximity and semantic similarity. To exploit geometric proximity, humans and objects are grouped into distinct clusters using a learnable proximity estimator based on spatial features derived from bounding boxes. In each group, a soft correspondence is computed via self-attention to aggregate and dispatch contextual cues. To incorporate semantic similarity, we enhance the vanilla transformer-based interaction decoder with semantic-aware local cues derived from HO-pair features. Extensive experiments on HICO-DET and V-COCO benchmarks demonstrate the superiority of GroupHOI over the state-of-the-art methods. The source code will be released.