Analyze Feature Flow to Enhance Interpretation and Steering in Language
  Models
We introduce a new approach to systematically map features discovered by
sparse autoencoder across consecutive layers of large language models,
extending earlier work that examined inter-layer feature links. By using a
data-free cosine similarity technique, we trace how specific features persist,
transform, or first appear at each stage. This method yields granular flow
graphs of feature evolution, enabling fine-grained interpretability and
mechanistic insights into model computations. Crucially, we demonstrate how
these cross-layer feature maps facilitate direct steering of model behavior by
amplifying or suppressing chosen features, achieving targeted thematic control
in text generation. Together, our findings highlight the utility of a causal,
cross-layer interpretability framework that not only clarifies how features
develop through forward passes but also provides new means for transparent
manipulation of large language models.
67a59c4e7ffacd843a56404a
Daniil Laptev
claimed_verified
2025-02-07T09:58:04.546Z
634c5f8cfb80cc6bcaf42c03
/avatars/1f37db0e70cbaf9707f4c8cbcee37ca0.svg
Daniil Laptev
user
dlaptev
67a59c4e7ffacd843a56404b
Nikita Balagansky
claimed_verified
2025-02-07T09:58:02.693Z
60b364e7f88532cd79eaff7b
https://cdn-avatars.huggingface.co/v1/production/uploads/1654185363389-60b364e7f88532cd79eaff7b.jpeg
Nikita Balagansky
user
elephantmipt
67a59c4e7ffacd843a56404c
Yaroslav Aksenov
claimed_verified
2025-02-13T20:36:09.037Z
63ed5676684767daecac6f8a
/avatars/d0e4a715f9c3fb6d74c183bab751ec35.svg
Yaroslav Aksenov
user
yaraksen
67a59c4e7ffacd843a56404d
Daniil Gavrilov
claimed_verified
2025-02-07T09:58:06.718Z
62a9c8edc19f92ae443ab37f
https://cdn-avatars.huggingface.co/v1/production/uploads/1669110208492-62a9c8edc19f92ae443ab37f.png
Daniil Gavrilov
user
kefirski
Analyze Feature Flow to Enhance Interpretation and Steering in Language
  Models
We introduce a new approach to systematically map features discovered by
sparse autoencoder across consecutive layers of large language models,
extending earlier work that examined inter-layer feature links. By using a
data-free cosine similarity technique, we trace how specific features persist,
transform, or first appear at each stage. This method yields granular flow
graphs of feature evolution, enabling fine-grained interpretability and
mechanistic insights into model computations. Crucially, we demonstrate how
these cross-layer feature maps facilitate direct steering of model behavior by
amplifying or suppressing chosen features, achieving targeted thematic control
in text generation. Together, our findings highlight the utility of a causal,
cross-layer interpretability framework that not only clarifies how features
develop through forward passes but also provides new means for transparent
manipulation of large language models.
sparse autoencoder large language models inter-layer feature links cosine similarity feature persistence feature transformation feature appearance flow graphs feature evolution interpretability mechanistic insights model computations cross-layer feature maps model behavior feature amplification feature suppression targeted thematic control text generation causal transparent manipulation