Enhancing Automated Interpretability with Output-Centric Feature
  Descriptions
Automated interpretability pipelines generate natural language descriptions
for the concepts represented by features in large language models (LLMs), such
as plants or the first word in a sentence. These descriptions are derived using
inputs that activate the feature, which may be a dimension or a direction in
the model's representation space. However, identifying activating inputs is
costly, and the mechanistic role of a feature in model behavior is determined
both by how inputs cause a feature to activate and by how feature activation
affects outputs. Using steering evaluations, we reveal that current pipelines
provide descriptions that fail to capture the causal effect of the feature on
outputs. To fix this, we propose efficient, output-centric methods for
automatically generating feature descriptions. These methods use the tokens
weighted higher after feature stimulation or the highest weight tokens after
applying the vocabulary "unembedding" head directly to the feature. Our
output-centric descriptions better capture the causal effect of a feature on
model outputs than input-centric descriptions, but combining the two leads to
the best performance on both input and output evaluations. Lastly, we show that
output-centric descriptions can be used to find inputs that activate features
previously thought to be "dead".
67876067c377690c01ab4478
Yoav Gur-Arieh
claimed_verified
2025-05-30T06:58:57.951Z
621febb6c7f47c5eb5df001d
/avatars/6096101dc01f1a7e39b1d0826170412d.svg
Yoav Gur Arieh
user
yoavgur
67876067c377690c01ab4479
Roy Mayan
admin_assigned
2025-01-15T16:59:28.079Z
66640b931284bd77155ea6cf
https://cdn-avatars.huggingface.co/v1/production/uploads/66640b931284bd77155ea6cf/pvhrHFLbr0vUPfuYGQT-Q.jpeg
Roy Mayan
user
roym44
67876067c377690c01ab447a
Chen Agassy
claimed_verified
2025-01-16T09:00:03.349Z
6788ca64f7306dbc1b757596
/avatars/5e0cee8de90372fbf8f61aecdc76d498.svg
Chen Agassy
user
chenagassy
67876067c377690c01ab447b
Atticus Geiger
extracted_confirmed
2025-01-15T15:36:25.790Z
627b2d0527dc4650b62eef42
/avatars/e70381850f5657b54e90f5539f3d74eb.svg
Atticus Geiger
user
atticusg
67876067c377690c01ab447c
Mor Geva
admin_assigned
2025-01-15T16:59:15.469Z
610b729f9da682cd54ad9adf
https://cdn-avatars.huggingface.co/v1/production/uploads/1628140189042-noauth.jpeg
Mor Geva
user
mega
Enhancing Automated Interpretability with Output-Centric Feature
  Descriptions
Automated interpretability pipelines generate natural language descriptions
for the concepts represented by features in large language models (LLMs), such
as plants or the first word in a sentence. These descriptions are derived using
inputs that activate the feature, which may be a dimension or a direction in
the model's representation space. However, identifying activating inputs is
costly, and the mechanistic role of a feature in model behavior is determined
both by how inputs cause a feature to activate and by how feature activation
affects outputs. Using steering evaluations, we reveal that current pipelines
provide descriptions that fail to capture the causal effect of the feature on
outputs. To fix this, we propose efficient, output-centric methods for
automatically generating feature descriptions. These methods use the tokens
weighted higher after feature stimulation or the highest weight tokens after
applying the vocabulary "unembedding" head directly to the feature. Our
output-centric descriptions better capture the causal effect of a feature on
model outputs than input-centric descriptions, but combining the two leads to
the best performance on both input and output evaluations. Lastly, we show that
output-centric descriptions can be used to find inputs that activate features
previously thought to be "dead".
Efficient, output-centric methods improve feature descriptions in large language models by better capturing causal effects and identifying previously inactive features.
automated interpretability pipelines large language models (LLMs) feature descriptions steering evaluations input-centric descriptions output-centric descriptions feature stimulation vocabulary "unembedding" head dead features