Beyond the Linear Separability Ceiling
Most state-of-the-art Visual-Language Models (VLMs) are seemingly limited by
the linear separabilty of their visual embeddings on abstract reasoning tasks.
This work investigates this "linear reasoning bottleneck" by introducing the
Linear Separability Ceiling (LSC), the performance of a simple linear
classifier on a VLM's visual embeddings. We find this bottleneck is widespread
and stems not from poor perception, but from failures in the language model's
reasoning pathways. We demonstrate this is a solvable alignment issue. The
required intervention, however, is task-dependent: activating existing pathways
suffices for semantic concepts, while complex relational reasoning requires
adapting core model weights. Using postfix tuning as a methodological control,
we find strong evidence for powerful, dormant reasoning pathways within VLMs.
However, for complex relational tasks requiring deeper adaptation, explicitly
improving representation quality causes the model to fail on new prompt formats
despite its embeddings remaining well separated. Ultimately, this work provides
a new lens for VLM analysis, showing that robust reasoning is a matter of
targeted alignment, not simply improved representation learning.
6870c2d3c8391850d60978e6
Enrico Vompa
claimed_verified
2025-07-11T07:59:54.331Z
63aadfe9a4bdd629b7ea7692
/avatars/9cc4eb5d4090ce84a590ec195b70e545.svg
Enrico
user
envomp
6870c2d3c8391850d60978e7
Tanel Tammet
6870c2d3c8391850d60978e8
Mohit Vaishnav
Beyond the Linear Separability Ceiling
Most state-of-the-art Visual-Language Models (VLMs) are seemingly limited by
the linear separabilty of their visual embeddings on abstract reasoning tasks.
This work investigates this "linear reasoning bottleneck" by introducing the
Linear Separability Ceiling (LSC), the performance of a simple linear
classifier on a VLM's visual embeddings. We find this bottleneck is widespread
and stems not from poor perception, but from failures in the language model's
reasoning pathways. We demonstrate this is a solvable alignment issue. The
required intervention, however, is task-dependent: activating existing pathways
suffices for semantic concepts, while complex relational reasoning requires
adapting core model weights. Using postfix tuning as a methodological control,
we find strong evidence for powerful, dormant reasoning pathways within VLMs.
However, for complex relational tasks requiring deeper adaptation, explicitly
improving representation quality causes the model to fail on new prompt formats
despite its embeddings remaining well separated. Ultimately, this work provides
a new lens for VLM analysis, showing that robust reasoning is a matter of
targeted alignment, not simply improved representation learning.
Visual-Language Models are limited by linear separability of visual embeddings in abstract reasoning tasks, which can be addressed through targeted alignment rather than improved representation learning.
Visual-Language Models Linear Separability Ceiling linear classifier visual embeddings linear reasoning bottleneck language model reasoning pathways postfix tuning representation quality prompt formats