Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoning
Yang Li, Aming WU, Zihao Zhang, Yahong Han
In this paper, we focus on Novel Class Discoveryfor Point Cloud Segmentation (3D-NCD), aiming to learn a model that can segment unlabeled(novel) 3D classes using only the supervisionfrom labeled (base) 3D classes. The key of the thistask is to setup the exact correlations between thepoint representations and their base class labels,as well as the representation correlations betweenthe points from base and novel classes. A coarseor statistical correlation learning may lead to theconfusion in novel class inference. lf we imposea casual relationship as a strong correlated constraint upon the learning process, the essentialpoint cloud representations that accurately correspond to the classes should be uncovered. Tothis end, we introduce a structural causal model(SCM) to re-formalize the 3D-NCD problem andpropose a new method, i.e., Joint Learning ofCausal Representation and Reasoning. Specifically, we first analyze hidden confounders in thebase class representations and the causal relationships between the base and novel classes throughSCM. We devise a causal representation prototypethat eliminates confounders to capture the causalrepresentations of base classes. A graph structureis then used to model the causal relationships between the base classes’ casual representation prototypes and the novel class prototypes, enablingcausal reasoning from base to novel classes. Extensive experiments and visualization results on3D and 2D NCD semantic segmentation demonstrate the superiorities of our method.