Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions
Hidde Fokkema, Tim van Erven, Sara Magliacane
Machine learning is a vital part of many real-world systems, but several concernsremain about the lack of interpretability, explainability and robustness of black-boxAI systems. Concept Bottleneck Models (CBM) address some of these challengesby learning interpretable concepts from high-dimensional data, e.g. images, whichare used to predict labels. An important issue in CBMs are spurious correlationbetween concepts, which effectively lead to learning “wrong” concepts. Currentmitigating strategies have strong assumptions, e.g., they assume that the conceptsare statistically independent of each other, or require substantial interaction interms of both interventions and labels provided by annotators. In this paper, wedescribe a framework that provides theoretical guarantees on the correctness ofthe learned concepts and on the number of required labels, without requiring anyinterventions. Our framework leverages causal representation learning (CRL)methods to learn latent causal variables from high-dimensional observations ina unsupervised way, and then learns to align these variables with interpretableconcepts with few concept labels. We propose a linear and a non-parametricestimator for this mapping, providing a finite-sample high probability result in thelinear case and an asymptotic consistency result for the non-parametric estimator.We evaluate our framework in synthetic and image benchmarks, showing that thelearned concepts have less impurities and are often more accurate than other CBMs,even in settings with strong correlations between concepts.