Not All Language Model Features Are Linear
Recent work has proposed the linear representation hypothesis: that language
models perform computation by manipulating one-dimensional representations of
concepts ("features") in activation space. In contrast, we explore whether some
language model representations may be inherently multi-dimensional. We begin by
developing a rigorous definition of irreducible multi-dimensional features
based on whether they can be decomposed into either independent or
non-co-occurring lower-dimensional features. Motivated by these definitions, we
design a scalable method that uses sparse autoencoders to automatically find
multi-dimensional features in GPT-2 and Mistral 7B. These auto-discovered
features include strikingly interpretable examples, e.g. circular features
representing days of the week and months of the year. We identify tasks where
these exact circles are used to solve computational problems involving modular
arithmetic in days of the week and months of the year. Finally, we provide
evidence that these circular features are indeed the fundamental unit of
computation in these tasks with intervention experiments on Mistral 7B and
Llama 3 8B, and we find further circular representations by breaking down the
hidden states for these tasks into interpretable components.
66501b5bd62f7e57e34d7ac8
Joshua Engels
extracted_confirmed
2024-05-24T10:58:22.151Z
6610bbffb5e6420d03bb3b05
/avatars/8ab37ccf48410f6fc9f0f4fa851060fe.svg
Josh Engels
user
JoshEngels
66501b5bd62f7e57e34d7ac9
Isaac Liao
66501b5bd62f7e57e34d7aca
Eric J. Michaud
extracted_confirmed
2024-05-24T11:59:59.324Z
6303e59c7b50dd9d0a367375
/avatars/353aaaae6db1c7dc89895927a65fa9b1.svg
Eric Michaud
user
ericjm
66501b5bd62f7e57e34d7acb
Wes Gurnee
extracted_confirmed
2024-05-24T05:58:45.489Z
631f446fdaa9591e52319035
/avatars/34d7ff8bc59aac0ed77a961596c4e4b1.svg
Wes Gurnee
user
wesg
66501b5bd62f7e57e34d7acc
Max Tegmark
Not All Language Model Features Are Linear
Recent work has proposed the linear representation hypothesis: that language
models perform computation by manipulating one-dimensional representations of
concepts ("features") in activation space. In contrast, we explore whether some
language model representations may be inherently multi-dimensional. We begin by
developing a rigorous definition of irreducible multi-dimensional features
based on whether they can be decomposed into either independent or
non-co-occurring lower-dimensional features. Motivated by these definitions, we
design a scalable method that uses sparse autoencoders to automatically find
multi-dimensional features in GPT-2 and Mistral 7B. These auto-discovered
features include strikingly interpretable examples, e.g. circular features
representing days of the week and months of the year. We identify tasks where
these exact circles are used to solve computational problems involving modular
arithmetic in days of the week and months of the year. Finally, we provide
evidence that these circular features are indeed the fundamental unit of
computation in these tasks with intervention experiments on Mistral 7B and
Llama 3 8B, and we find further circular representations by breaking down the
hidden states for these tasks into interpretable components.
Research explores multi-dimensional features in language models, discovering interpretable circular representations in GPT-2, Mistral 7B, and Llama 3 8B, which are used for modular arithmetic tasks.
linear representation hypothesis multi-dimensional features irreducible multi-dimensional features sparse autoencoders GPT-2 Mistral 7B circular features modular arithmetic intervention experiments Llama 3 8B