Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?
Yihao Li, Saeed Salehi, Lyle Ungar, Konrad Kording
Object binding, the brain’s ability to bind the many features that collectively represent an object into a coherent whole, is central to human cognition. It groups low-level perceptual features into high‑level object representations, stores those objects efficiently and compositionally in memory, and supports human reasoning about individual object instances. While prior work often imposes object-centric attention (e.g., Slot Attention) explicitly to probe these benefits, it remains unclear whether this ability naturally emerges in pre-trained Vision Transformers (ViTs). Intuitively, they could: recognizing which patches belong to the same object should be useful for downstream prediction and thus guide attention. We hypothesize that these models thus represent whether two patches belong to the same object, a property we term $IsSameObject$, motivated by the pairwise nature of their self-attention mechanism. Here, we ask whether pre-trained ViTs spontaneously develop object binding. To test this, we decode $IsSameObject$ from patch embeddings across the hierarchy of layers in the ViT using a similarity probe. We demonstrate that our probe achieves over 90\% accuracy, outperforming pointwise probes that directly decode object type (85.7\%) or object identity (82.1\%), and this representation gradually emerges through the hierarchy. We further discover a low-dimensional subspace that encodes $IsSameObject$ on top of object features and show that early‑layer attention weights correlate strongly with this signal, which explains the utility of $IsSameObject$ representations. Ablating $IsSameObject$ from model activations degrades downstream performance and conflicts with self‑supervised pre-training loss minimization, implying that emergent object binding naturally serves the pretraining objective. Our findings challenge the view that ViTs lack object binding and highlight how symbolic knowledge of “which parts belong together” emerges naturally in a connectionist system.