Monet: Mixture of Monosemantic Experts for Transformers
Understanding the internal computations of large language models (LLMs) is
crucial for aligning them with human values and preventing undesirable
behaviors like toxic content generation. However, mechanistic interpretability
is hindered by polysemanticity -- where individual neurons respond to multiple,
unrelated concepts. While Sparse Autoencoders (SAEs) have attempted to
disentangle these features through sparse dictionary learning, they have
compromised LLM performance due to reliance on post-hoc reconstruction loss. To
address this issue, we introduce Mixture of Monosemantic Experts for
Transformers (Monet) architecture, which incorporates sparse dictionary
learning directly into end-to-end Mixture-of-Experts pretraining. Our novel
expert decomposition method enables scaling the expert count to 262,144 per
layer while total parameters scale proportionally to the square root of the
number of experts. Our analyses demonstrate mutual exclusivity of knowledge
across experts and showcase the parametric knowledge encapsulated within
individual experts. Moreover, Monet allows knowledge manipulation over domains,
languages, and toxicity mitigation without degrading general performance. Our
pursuit of transparent LLMs highlights the potential of scaling expert counts
to enhance} mechanistic interpretability and directly resect the internal
knowledge to fundamentally adjust} model behavior. The source code and
pretrained checkpoints are available at https://github.com/dmis-lab/Monet.
675282e85b1cc8f5a17f70d2
Jungwoo Park
claimed_verified
2024-12-06T09:08:53.485Z
60f8435644e75317cc02ed51
/avatars/68b7fc077fe2bda6607b1c470add8140.svg
Jungwoo Park
user
affjljoo3581
675282e85b1cc8f5a17f70d3
Young Jin Ahn
claimed_verified
2024-12-09T11:07:18.378Z
613f076fd86a0afab8db7117
https://cdn-avatars.huggingface.co/v1/production/uploads/1637046229907-613f076fd86a0afab8db7117.jpeg
Ahn Young Jin
user
snoop2head
675282e85b1cc8f5a17f70d4
Kee-Eung Kim
675282e85b1cc8f5a17f70d5
Jaewoo Kang
Monet: Mixture of Monosemantic Experts for Transformers
Understanding the internal computations of large language models (LLMs) is
crucial for aligning them with human values and preventing undesirable
behaviors like toxic content generation. However, mechanistic interpretability
is hindered by polysemanticity -- where individual neurons respond to multiple,
unrelated concepts. While Sparse Autoencoders (SAEs) have attempted to
disentangle these features through sparse dictionary learning, they have
compromised LLM performance due to reliance on post-hoc reconstruction loss. To
address this issue, we introduce Mixture of Monosemantic Experts for
Transformers (Monet) architecture, which incorporates sparse dictionary
learning directly into end-to-end Mixture-of-Experts pretraining. Our novel
expert decomposition method enables scaling the expert count to 262,144 per
layer while total parameters scale proportionally to the square root of the
number of experts. Our analyses demonstrate mutual exclusivity of knowledge
across experts and showcase the parametric knowledge encapsulated within
individual experts. Moreover, Monet allows knowledge manipulation over domains,
languages, and toxicity mitigation without degrading general performance. Our
pursuit of transparent LLMs highlights the potential of scaling expert counts
to enhance} mechanistic interpretability and directly resect the internal
knowledge to fundamentally adjust} model behavior. The source code and
pretrained checkpoints are available at https://github.com/dmis-lab/Monet.
The Mixture of Monosemantic Experts for Transformers (Monet) architecture enhances mechanistic interpretability of large language models by incorporating sparse dictionary learning directly into end-to-end pretraining, allowing for increased expert count and improved knowledge manipulation.
Sparse Autoencoders Mixture-of-Experts pretraining mutual exclusivity of knowledge parametric knowledge toxicity mitigation mechanistic interpretability knowledge manipulation