Can sparse autoencoders be used to decompose and interpret steering
  vectors?
Steering vectors are a promising approach to control the behaviour of large
language models. However, their underlying mechanisms remain poorly understood.
While sparse autoencoders (SAEs) may offer a potential method to interpret
steering vectors, recent findings show that SAE-reconstructed vectors often
lack the steering properties of the original vectors. This paper investigates
why directly applying SAEs to steering vectors yields misleading
decompositions, identifying two reasons: (1) steering vectors fall outside the
input distribution for which SAEs are designed, and (2) steering vectors can
have meaningful negative projections in feature directions, which SAEs are not
designed to accommodate. These limitations hinder the direct use of SAEs for
interpreting steering vectors.
6735c8b06d4b6c741016a221
Harry Mayne
admin_assigned
2024-11-15T14:19:33.411Z
63f887bcdf053017d1ad2967
https://cdn-avatars.huggingface.co/v1/production/uploads/63f887bcdf053017d1ad2967/bJ4KEqIK7TeG8iWOR7W2x.jpeg
Harry Mayne
user
HarryMayne
6735c8b06d4b6c741016a222
Yushi Yang
admin_assigned
2024-11-15T14:23:16.505Z
6562737bf5532ac1bd09d3bd
/avatars/aaaff49b6abb5f9351159006b7755d25.svg
YY
user
yy0514
6735c8b06d4b6c741016a223
Adam Mahdi
claimed_verified
2024-11-20T10:09:14.308Z
673cf9c93af47d1d2b6584c9
/avatars/0792921a8adaa1493255abb0b5aaa4f1.svg
Adam Mahdi
user
ammaox
Can sparse autoencoders be used to decompose and interpret steering
  vectors?
Steering vectors are a promising approach to control the behaviour of large
language models. However, their underlying mechanisms remain poorly understood.
While sparse autoencoders (SAEs) may offer a potential method to interpret
steering vectors, recent findings show that SAE-reconstructed vectors often
lack the steering properties of the original vectors. This paper investigates
why directly applying SAEs to steering vectors yields misleading
decompositions, identifying two reasons: (1) steering vectors fall outside the
input distribution for which SAEs are designed, and (2) steering vectors can
have meaningful negative projections in feature directions, which SAEs are not
designed to accommodate. These limitations hinder the direct use of SAEs for
interpreting steering vectors.
Investigation into why sparse autoencoders yield misleading decompositions of steering vectors in large language models reveals two main limitations.
sparse autoencoders steering vectors input distribution negative projections feature directions