Correcting misinterpretations in additive models
Benedict Clark, Rick Wilming, Hjalmar Schulz, Rustam Zhumagambetov, Danny Panknin, Stefan Haufe
Correct model interpretation in high-stakes settings is critical, yet both post-hoc explanation methods and so-called intrinsically interpretable models can systematically attribute false-positive importance to non-informative features such as suppressor variables (Wilming et al., 2023).Specifically, both linear models and their powerful non-linear generalisation such as Neural Additive Models (NAMs, Agarwal et al., 2021) are susceptible to spurious attributions to suppressors. We present a principled generalisation of activation patterns -- originally developed to make linear models truly interpretable -- to additive models, introducing a covariance-aware explanation framework that correctly disentangles suppressor effects.This yields PatternGAM, an activation-based explanation method for the broad range of additive models that also includes NAMs, and PatternQLR for polynomial logistic regression models. These provide meaningful and reliable attributions to truly informative features in the presence of complex input dependencies, and especially in scenarios where pairwise interaction terms are required. Empirical evaluations on the XAI-TRIS benchmark (Clark et al., 2024) with a novel false-negative invariant formulation of the earth mover's distance metric demonstrates significant improvements over popular XAI methods and the traditional interpretation of additive models. Finally, real-world case studies on the COMPAS (ProPublica, 2016) and MIMIC-IV (Johnson et al., 2023b) datasets confirm that our approach aligns with domain knowledge and provides new insights into the role of specific features by disentangling genuine target-related information from suppressor effects.