Integral Imprecise Probability Metrics
Siu Lun Chau, Michele Caprio, Krikamol Muandet
Quantifying differences between probability measures is central to statistics and machine learning. As interest in Imprecise Probabilistic Machine Learning (IPML) grows---where inference and decision-making are grounded in richer representations of uncertainty with imprecise probabilities (IP)---there is an increasing need for metrics beyond classical probability. This work introduces the Integral Imprecise Probability Metric (IIPM) framework, a Choquet integral-based generalisation of classical Integral Probability Metric (IPM) to the setting of capacities---a broad class of IP models encompassing many existing ones, including lower probabilities, probability intervals, belief functions, and more. Theoretically, we establish conditions under which IIPM serves as a valid metric and metrises a form of weak convergence of capacities. Practically, IIPM not only enables comparison across different IP models but also supports the quantification of epistemic uncertainty (EU) within a single IP model. In particular, by comparing an IP model with its conjugate, IIPM gives rise to a new class of EU measures---Maximum Mean Imprecisions (MMIs)---which satisfy key axiomatic properties proposed in the uncertainty quantification (UQ) literature. We validate MMI through selective classification experiments, demonstrating strong empirical performance against established EU measures, and outperforming them when classical methods struggle to scale to a large number of classes. Our work advances both theory and practice in IPML, offering a principled framework for comparing and quantifying epistemic uncertainty under imprecision.