Aggregation Hides OOD Generalization Failures from Spurious Correlations
Olawale Salaudeen, Haoran Zhang, Kumail Alhamoud, Sara Beery, Marzyeh Ghassemi
Benchmarks for out‑of‑distribution (OOD) generalization frequently show a strong positive correlation between in‑distribution (ID) and OOD accuracy, termed "accuracy‑on‑the‑line." This pattern is often taken to imply that spurious correlations---correlations that improve ID but reduce OOD performance---are rare in practice. We find that this positive correlation is an artifact of aggregating heterogeneous OOD examples. Using a simple gradient‑based method, we partition each benchmark’s OOD split into semantically coherent subsets where accuracy on the line does not hold. Across six widely used distribution shift benchmarks, the method uncovers subsets, sometimes up to 77% of the usual OOD split, where higher ID accuracy predicts lower OOD accuracy. Our findings indicate that aggregate metrics can obscure important failure modes of OOD robustness. We release code and the identified subsets to facilitate further evaluation.