Reasoning Beyond Points: A Visual Introspective Approach for Few-Shot 3D Segmentation
Changshuo Wang, Shuting He, Xiang Fang, Zhijian Hu, JIA-HONG HUANG, Yixian Shen, Prayag Tiwari
Point Cloud Few-Shot Semantic Segmentation (PC-FSS) aims to segment unknown categories in query samples using only a small number of annotated support samples. However, scene complexity and insufficient representation of local geometric structures pose significant challenges to PC-FSS. To address these issues, we propose a novel pre-training-free Visual Introspective Prototype Segmentation network (VIP-Seg). Specifically, inspired by Chain-of-Thought reasoning, we design a Visual Introspective Prototype (VIP) module to tackle intra-class diversity and domain gaps between support and query sets. The VIP module consists of a Prototype Enhancement Module (PEM) and a Prototype Difference Module (PDM), which work alternately to form a multi-step reasoning process. The PEM enhances prototype discriminability and reduces intra-class diversity, while the PDM learns common representations from the differences between query and support features, effectively eliminating semantic inconsistencies caused by domain gaps. To further reduce intra-class diversity and enhance point discriminative ability, we propose a Dynamic Taylor Convolution (DyTaylorConv), inspired by the Taylor series, which effectively captures local geometric structures and detailed features of point clouds. Extensive experiments on S3DIS and ScanNet demonstrate that our proposed VIP-Seg significantly outperforms current state-of-the-art methods, proving the its effectiveness in PC-FSS tasks.