Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse
decomposition of a neural network's latent representations into seemingly
interpretable features. Despite recent excitement about their potential,
research applications outside of industry are limited by the high cost of
training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope,
an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2
2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs
on the Gemma 2 pre-trained models, but additionally release SAEs trained on
instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each
SAE on standard metrics and release these results. We hope that by releasing
these SAE weights, we can help make more ambitious safety and interpretability
research easier for the community. Weights and a tutorial can be found at
https://huggingface.co/google/gemma-scope and an interactive demo can be found
at https://www.neuronpedia.org/gemma-scope
66b97edaa0ebe4b8a50f1b44
Tom Lieberum
66b97edaa0ebe4b8a50f1b45
Senthooran Rajamanoharan
admin_assigned
2024-08-13T09:58:33.347Z
6477122c99a5ce743ccf2f55
/avatars/dee617374fc609e07eba3bcb2cd16810.svg
Senthooran Rajamanoharan
user
srdm
66b97edaa0ebe4b8a50f1b46
Arthur Conmy
claimed_verified
2024-08-13T07:38:33.046Z
631ecc58daa9591e522e1494
/avatars/4f808fae966e808105e89712c97d90d2.svg
VConm
user
ArthurConmy
66b97edaa0ebe4b8a50f1b47
Lewis Smith
66b97edaa0ebe4b8a50f1b48
Nicolas Sonnerat
66b97edaa0ebe4b8a50f1b49
Vikrant Varma
admin_assigned
2024-08-13T09:59:12.295Z
667f25c67a2adf8ac1c940b2
/avatars/ccf62e2e4913d9a8615613f7bb0bfbe2.svg
Vikrant Varma
user
vikrantvarma
66b97edaa0ebe4b8a50f1b4a
János Kramár
admin_assigned
2024-08-13T09:59:22.691Z
6647488b45ee22196e798ba6
/avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
Janos Kramar
user
jkramar
66b97edaa0ebe4b8a50f1b4b
Anca Dragan
66b97edaa0ebe4b8a50f1b4c
Rohin Shah
66b97edaa0ebe4b8a50f1b4d
Neel Nanda
admin_assigned
2024-08-13T10:00:03.588Z
62669380c8bc5cf80ca97350
/avatars/6d5cd2261163308b82341c1ce28984d1.svg
Neel Nanda
user
NeelNanda
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse
decomposition of a neural network's latent representations into seemingly
interpretable features. Despite recent excitement about their potential,
research applications outside of industry are limited by the high cost of
training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope,
an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2
2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs
on the Gemma 2 pre-trained models, but additionally release SAEs trained on
instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each
SAE on standard metrics and release these results. We hope that by releasing
these SAE weights, we can help make more ambitious safety and interpretability
research easier for the community. Weights and a tutorial can be found at
https://huggingface.co/google/gemma-scope and an interactive demo can be found
at https://www.neuronpedia.org/gemma-scope
Gemma Scope is an open suite of JumpReLU SAEs trained on various layers of Gemma 2 models to facilitate safety and interpretability research.
sparse autoencoders JumpReLU latent representations Gemma 2 instruction-tuned SAE weights