Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language
  Model
Generalized few-shot 3D point cloud segmentation (GFS-PCS) adapts models to
new classes with few support samples while retaining base class segmentation.
Existing GFS-PCS methods enhance prototypes via interacting with support or
query features but remain limited by sparse knowledge from few-shot samples.
Meanwhile, 3D vision-language models (3D VLMs), generalizing across open-world
novel classes, contain rich but noisy novel class knowledge. In this work, we
introduce a GFS-PCS framework that synergizes dense but noisy pseudo-labels
from 3D VLMs with precise yet sparse few-shot samples to maximize the strengths
of both, named GFS-VL. Specifically, we present a prototype-guided pseudo-label
selection to filter low-quality regions, followed by an adaptive infilling
strategy that combines knowledge from pseudo-label contexts and few-shot
samples to adaptively label the filtered, unlabeled areas. Additionally, we
design a novel-base mix strategy to embed few-shot samples into training
scenes, preserving essential context for improved novel class learning.
Moreover, recognizing the limited diversity in current GFS-PCS benchmarks, we
introduce two challenging benchmarks with diverse novel classes for
comprehensive generalization evaluation. Experiments validate the effectiveness
of our framework across models and datasets. Our approach and benchmarks
provide a solid foundation for advancing GFS-PCS in the real world. The code is
at https://github.com/ZhaochongAn/GFS-VL
67e13fcab21a55baf5fc6768
Zhaochong An
claimed_verified
2025-03-24T12:32:41.985Z
65e5eae6958b39864e8b683e
/avatars/b6a857e7b725767197dd95bc876f8ad1.svg
Zhaochong An
user
ZhaochongAn
67e13fcab21a55baf5fc6769
Guolei Sun
admin_assigned
2025-03-24T12:55:55.488Z
653c2b3777c2f094527b7a9c
/avatars/b27ac82a79a2d62cbecda303239efc2c.svg
Guolei Sun
user
GuoleiSun
67e13fcab21a55baf5fc676a
Yun Liu
67e13fcab21a55baf5fc676b
Runjia Li
admin_assigned
2025-03-24T12:56:19.776Z
638e29cf319f9c746b87ad4b
/avatars/70cac8d47847c389eb0393051a64c4a4.svg
Runjia Li
user
liguang0115
67e13fcab21a55baf5fc676c
Junlin Han
admin_assigned
2025-03-24T12:56:41.798Z
636e6ee287545ca5a136b4c3
/avatars/208d32b1202e2da210146027212dbdd3.svg
Junlin Han
user
Junlinh
67e13fcab21a55baf5fc676d
Ender Konukoglu
67e13fcab21a55baf5fc676e
Serge Belongie
Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language
  Model
Generalized few-shot 3D point cloud segmentation (GFS-PCS) adapts models to
new classes with few support samples while retaining base class segmentation.
Existing GFS-PCS methods enhance prototypes via interacting with support or
query features but remain limited by sparse knowledge from few-shot samples.
Meanwhile, 3D vision-language models (3D VLMs), generalizing across open-world
novel classes, contain rich but noisy novel class knowledge. In this work, we
introduce a GFS-PCS framework that synergizes dense but noisy pseudo-labels
from 3D VLMs with precise yet sparse few-shot samples to maximize the strengths
of both, named GFS-VL. Specifically, we present a prototype-guided pseudo-label
selection to filter low-quality regions, followed by an adaptive infilling
strategy that combines knowledge from pseudo-label contexts and few-shot
samples to adaptively label the filtered, unlabeled areas. Additionally, we
design a novel-base mix strategy to embed few-shot samples into training
scenes, preserving essential context for improved novel class learning.
Moreover, recognizing the limited diversity in current GFS-PCS benchmarks, we
introduce two challenging benchmarks with diverse novel classes for
comprehensive generalization evaluation. Experiments validate the effectiveness
of our framework across models and datasets. Our approach and benchmarks
provide a solid foundation for advancing GFS-PCS in the real world. The code is
at https://github.com/ZhaochongAn/GFS-VL
A generalized few-shot 3D point cloud segmentation framework integrates noisy pseudo-labels from 3D vision-language models with sparse few-shot samples to enhance segmentation accuracy and generalization.
few-shot 3D point cloud segmentation GFS-PCS 3D vision-language models 3D VLMs prototype-guided pseudo-label selection adaptive infilling strategy novel-base mix strategy benchmarks