Representational Geometry Collapse in ANNs Limits Semantic Generalization: A Dimensionality Mismatch Between Brains and ANNs
Santiago Galella, Maren Wehrheim, Matthias Kaschube
Biological and artificial vision systems both rely on hierarchical architectures, yet it remains unclear how their representational geometry evolves across processing stages, and what functional consequences may arise from potential differences. In this work, we systematically quantify and compare the linear and non-linear (intrinsic) dimensionality of human brain activity (fMRI) and artificial neural networks (ANNs) during natural image viewing. In the human ventral visual stream, both dimensionality measures increase super-linearly along the visual hierarchy, supporting the emergence of semantic and abstract representations. For linear dimensionality, ANNs show a similar increase, but only for pooled features, reinforcing the importance of appropriate feature readouts in brain–model comparisons. In contrast, nonlinear dimensionality shows a collapse in the later layers of ANNs, pointing at a mismatch in representational geometry between the human and artificial visual systems. Moreover, we show that this mismatch may have important functional consequences: high-dimensional brain representations support flexible generalization to abstract, behaviorally rated features, while ANNs lose access to such information in deeper layers despite achieving high categorization accuracy. We show that in the brain, dimensionality increases along the visual hierarchy and supports the encoding of abstract features. In contrast, ANNs exhibit late-stage compression, reflected by the collapse of nonlinear dimensionality, which appears to limit their capacity for abstraction. Our findings suggest that representational geometry provides a functional signature of the ability of abstraction, and propose dimensionality alignment as a benchmark for building more flexible and biologically grounded vision models.