Multimodal Neurons in Pretrained Text-Only Transformers
Language models demonstrate remarkable capacity to generalize representations
learned in one modality to downstream tasks in other modalities. Can we trace
this ability to individual neurons? We study the case where a frozen text
transformer is augmented with vision using a self-supervised visual encoder and
a single linear projection learned on an image-to-text task. Outputs of the
projection layer are not immediately decodable into language describing image
content; instead, we find that translation between modalities occurs deeper
within the transformer. We introduce a procedure for identifying "multimodal
neurons" that convert visual representations into corresponding text, and
decoding the concepts they inject into the model's residual stream. In a series
of experiments, we show that multimodal neurons operate on specific visual
concepts across inputs, and have a systematic causal effect on image
captioning.
64cc5f898a16b1748f21b910
Sarah Schwettmann
admin_assigned
2023-08-04T12:43:18.131Z
648bbafe09f73fe180a5cf38
https://cdn-avatars.huggingface.co/v1/production/uploads/648bbafe09f73fe180a5cf38/BgNBoZequvWtp3ycBxy2M.png
Sarah Schwettmann
user
generish
64cc5f898a16b1748f21b911
Neil Chowdhury
claimed_verified
2023-08-07T12:38:49.615Z
63ec754359a46fd5f4b81a2f
/avatars/aaa4a63b25bd03c366f855427ba9cad4.svg
Neil Chowdhury
user
nch0w
64cc5f898a16b1748f21b912
Antonio Torralba
Multimodal Neurons in Pretrained Text-Only Transformers
Language models demonstrate remarkable capacity to generalize representations
learned in one modality to downstream tasks in other modalities. Can we trace
this ability to individual neurons? We study the case where a frozen text
transformer is augmented with vision using a self-supervised visual encoder and
a single linear projection learned on an image-to-text task. Outputs of the
projection layer are not immediately decodable into language describing image
content; instead, we find that translation between modalities occurs deeper
within the transformer. We introduce a procedure for identifying "multimodal
neurons" that convert visual representations into corresponding text, and
decoding the concepts they inject into the model's residual stream. In a series
of experiments, we show that multimodal neurons operate on specific visual
concepts across inputs, and have a systematic causal effect on image
captioning.
Frozen text transformers can incorporate visual information through self-supervised learning, with specific neurons facilitating translation between modalities and influencing image captioning.
text transformer self-supervised visual encoder linear projection multimodal neurons residual stream image captioning