Selective Contrastive Learning for Weakly Supervised Affordance
  Grounding
Facilitating an entity's interaction with objects requires accurately
identifying parts that afford specific actions. Weakly supervised affordance
grounding (WSAG) seeks to imitate human learning from third-person
demonstrations, where humans intuitively grasp functional parts without needing
pixel-level annotations. To achieve this, grounding is typically learned using
a shared classifier across images from different perspectives, along with
distillation strategies incorporating part discovery process. However, since
affordance-relevant parts are not always easily distinguishable, models
primarily rely on classification, often focusing on common class-specific
patterns that are unrelated to affordance. To address this limitation, we move
beyond isolated part-level learning by introducing selective prototypical and
pixel contrastive objectives that adaptively learn affordance-relevant cues at
both the part and object levels, depending on the granularity of the available
information. Initially, we find the action-associated objects in both
egocentric (object-focused) and exocentric (third-person example) images by
leveraging CLIP. Then, by cross-referencing the discovered objects of
complementary views, we excavate the precise part-level affordance clues in
each perspective. By consistently learning to distinguish affordance-relevant
regions from affordance-irrelevant background context, our approach effectively
shifts activation from irrelevant areas toward meaningful affordance cues.
Experimental results demonstrate the effectiveness of our method. Codes are
available at github.com/hynnsk/SelectiveCL.
689c9309fab6fdd2e52ac9fe
WonJun Moon
claimed_verified
2025-08-25T08:27:59.549Z
637b9711086af1cb122e99b3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ZyMIY8L2AmE7tE07A7gS9.png
WonJun Moon
user
WJ0830
689c9309fab6fdd2e52ac9ff
Hyun Seok Seong
claimed_verified
2025-08-14T13:38:24.925Z
689b448ed7a19791dd3c7912
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/8GZpAYFVtAB9u0YSuRKl2.png
Hyun Seok Seong
user
hynnsk
689c9309fab6fdd2e52aca00
Jae-Pil Heo
Selective Contrastive Learning for Weakly Supervised Affordance
  Grounding
Facilitating an entity's interaction with objects requires accurately
identifying parts that afford specific actions. Weakly supervised affordance
grounding (WSAG) seeks to imitate human learning from third-person
demonstrations, where humans intuitively grasp functional parts without needing
pixel-level annotations. To achieve this, grounding is typically learned using
a shared classifier across images from different perspectives, along with
distillation strategies incorporating part discovery process. However, since
affordance-relevant parts are not always easily distinguishable, models
primarily rely on classification, often focusing on common class-specific
patterns that are unrelated to affordance. To address this limitation, we move
beyond isolated part-level learning by introducing selective prototypical and
pixel contrastive objectives that adaptively learn affordance-relevant cues at
both the part and object levels, depending on the granularity of the available
information. Initially, we find the action-associated objects in both
egocentric (object-focused) and exocentric (third-person example) images by
leveraging CLIP. Then, by cross-referencing the discovered objects of
complementary views, we excavate the precise part-level affordance clues in
each perspective. By consistently learning to distinguish affordance-relevant
regions from affordance-irrelevant background context, our approach effectively
shifts activation from irrelevant areas toward meaningful affordance cues.
Experimental results demonstrate the effectiveness of our method. Codes are
available at github.com/hynnsk/SelectiveCL.
The method uses selective prototypical and pixel contrastive objectives to learn affordance-relevant cues from third-person demonstrations, improving upon traditional weakly supervised affordance grounding by focusing on both part and object levels.
weakly supervised affordance grounding WSAG CLIP prototypical objectives pixel contrastive objectives affordance-relevant cues affordance-irrelevant background context egocentric images exocentric images