Navigating the MIL Trade-Off: Flexible Pooling for Whole-Slide Image Classification
Hossein Jafarinia, Danial Hamdi, Amirhossein Alamdar, Elahe Zahiri, Soroush Vafaie Tabar, Alireza Alipanah, Nahal Mirzaie, Saeed Razavi, Amir Najafi, Mohammad Hossein Rohban
Multiple Instance Learning (MIL) is a standard weakly supervised approach for Whole Slide Image (WSI) classification, where performance hinges on both feature representation and MIL-pooling strategies. Recent research has predominantly focused on Transformer-based architectures adapted for WSIs. However, we argue that this trend faces a fundamental limitation: data scarcity. In typical settings, Transformer models yield only marginal gains without access to large-scale datasets—resources that are virtually inaccessible to all but a few well-funded research labs. This limitation motivates the exploration of simpler, non-attention-based methods in conjunction with unsupervised representation learning. Building on this motivation, we first present a series of theoretical results characterizing classical MIL-pooling strategies, revealing trade-offs between \emph{generalization} and \emph{sensitivity}. Grounded in this analysis, we introduce MaxSoft—a novel MIL-pooling function that enables flexible control over this trade-off, allowing adaptation to specific tasks and datasets. To further tackle real-world deployment challenges such as specimen heterogeneity, we propose PerPatch augmentation—a simple yet effective technique that enhances model robustness. Empirically, MaxSoft achieves state-of-the-art performance in low-data regimes across four major benchmarks, often matching or surpassing large-scale foundation models. When combined with PerPatch augmentation, this performance is further improved through increased robustness.