A Multimodal Automated Interpretability Agent
This paper describes MAIA, a Multimodal Automated Interpretability Agent.
MAIA is a system that uses neural models to automate neural model understanding
tasks like feature interpretation and failure mode discovery. It equips a
pre-trained vision-language model with a set of tools that support iterative
experimentation on subcomponents of other models to explain their behavior.
These include tools commonly used by human interpretability researchers: for
synthesizing and editing inputs, computing maximally activating exemplars from
real-world datasets, and summarizing and describing experimental results.
Interpretability experiments proposed by MAIA compose these tools to describe
and explain system behavior. We evaluate applications of MAIA to computer
vision models. We first characterize MAIA's ability to describe (neuron-level)
features in learned representations of images. Across several trained models
and a novel dataset of synthetic vision neurons with paired ground-truth
descriptions, MAIA produces descriptions comparable to those generated by
expert human experimenters. We then show that MAIA can aid in two additional
interpretability tasks: reducing sensitivity to spurious features, and
automatically identifying inputs likely to be mis-classified.
66271ebbc25c3b1ac48eec87
Tamar Rott Shaham
admin_assigned
2024-04-23T08:13:59.030Z
63914afeb6b839bb6143a6be
/avatars/da25b555d105f4755c8187479469ca77.svg
Tamar Rott Shaham
user
tamarott
66271ebbc25c3b1ac48eec88
Sarah Schwettmann
claimed_verified
2024-04-23T07:13:52.276Z
648bbafe09f73fe180a5cf38
https://cdn-avatars.huggingface.co/v1/production/uploads/648bbafe09f73fe180a5cf38/BgNBoZequvWtp3ycBxy2M.png
Sarah Schwettmann
user
generish
66271ebbc25c3b1ac48eec89
Franklin Wang
admin_assigned
2024-04-23T08:16:02.993Z
6537ec2f33b10aa38f3f6469
/avatars/f25b2dfb2dc4f03b245761dccf3df2d7.svg
Franklin Wang
user
fxwang
66271ebbc25c3b1ac48eec8a
Achyuta Rajaram
admin_assigned
2024-04-23T08:16:09.787Z
649ee2b51f44b2c0301d6be7
/avatars/b449f2b7b4545a034ce6f90cc2f5020d.svg
Achyuta Rajaram
user
flybamboo
66271ebbc25c3b1ac48eec8b
Evan Hernandez
admin_assigned
2024-04-23T08:16:26.196Z
644467a679e7797ce71d1980
/avatars/a0946ddb7f15cbbea123cacde76eb1aa.svg
Evan Hernandez
user
evanhernandez
66271ebbc25c3b1ac48eec8c
Jacob Andreas
66271ebbc25c3b1ac48eec8d
Antonio Torralba
A Multimodal Automated Interpretability Agent
This paper describes MAIA, a Multimodal Automated Interpretability Agent.
MAIA is a system that uses neural models to automate neural model understanding
tasks like feature interpretation and failure mode discovery. It equips a
pre-trained vision-language model with a set of tools that support iterative
experimentation on subcomponents of other models to explain their behavior.
These include tools commonly used by human interpretability researchers: for
synthesizing and editing inputs, computing maximally activating exemplars from
real-world datasets, and summarizing and describing experimental results.
Interpretability experiments proposed by MAIA compose these tools to describe
and explain system behavior. We evaluate applications of MAIA to computer
vision models. We first characterize MAIA's ability to describe (neuron-level)
features in learned representations of images. Across several trained models
and a novel dataset of synthetic vision neurons with paired ground-truth
descriptions, MAIA produces descriptions comparable to those generated by
expert human experimenters. We then show that MAIA can aid in two additional
interpretability tasks: reducing sensitivity to spurious features, and
automatically identifying inputs likely to be mis-classified.
MAIA, a multimodal automated interpretability agent, uses neural models to perform feature interpretation and failure mode discovery for other models, demonstrating comparable results to human experimenters and aiding in reducing sensitivity to spurious features and identifying potential misclassifications.
neural models feature interpretation failure mode discovery vision-language model neuron-level features learned representations ground-truth descriptions spurious features mis-classified inputs