Backpack Language Models
We present Backpacks: a new neural architecture that marries strong modeling
performance with an interface for interpretability and control. Backpacks learn
multiple non-contextual sense vectors for each word in a vocabulary, and
represent a word in a sequence as a context-dependent, non-negative linear
combination of sense vectors in this sequence. We find that, after training,
sense vectors specialize, each encoding a different aspect of a word. We can
interpret a sense vector by inspecting its (non-contextual, linear) projection
onto the output space, and intervene on these interpretable hooks to change the
model's behavior in predictable ways. We train a 170M-parameter Backpack
language model on OpenWebText, matching the loss of a GPT-2 small
(124Mparameter) Transformer. On lexical similarity evaluations, we find that
Backpack sense vectors outperform even a 6B-parameter Transformer LM's word
embeddings. Finally, we present simple algorithms that intervene on sense
vectors to perform controllable text generation and debiasing. For example, we
can edit the sense vocabulary to tend more towards a topic, or localize a
source of gender bias to a sense vector and globally suppress that sense.
64741ba97be71eb8b1ba55f6
John Hewitt
claimed_verified
2023-06-01T05:54:16.073Z
646ec1161a198f8520f3c705
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/QPCRraySl9zTnf7eG5ZJk.png
John Hewitt
user
johnhew
64741ba97be71eb8b1ba55f7
John Thickstun
admin_assigned
2023-05-30T08:59:50.043Z
619828ff465b7973a532dee6
https://cdn-avatars.huggingface.co/v1/production/uploads/619828ff465b7973a532dee6/A55HdKmHcZ9w6BxN-cGoE.jpeg
John Thickstun
user
jthickstun
64741ba97be71eb8b1ba55f8
Christopher D. Manning
admin_assigned
2023-05-30T08:59:30.963Z
5f555a1f8bf55658acfed19e
https://cdn-avatars.huggingface.co/v1/production/uploads/1628695407761-5f555a1f8bf55658acfed19e.jpeg
Christopher Manning
user
manning
64741ba97be71eb8b1ba55f9
Percy Liang
admin_assigned
2023-05-30T08:59:15.269Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
Backpack Language Models
We present Backpacks: a new neural architecture that marries strong modeling
performance with an interface for interpretability and control. Backpacks learn
multiple non-contextual sense vectors for each word in a vocabulary, and
represent a word in a sequence as a context-dependent, non-negative linear
combination of sense vectors in this sequence. We find that, after training,
sense vectors specialize, each encoding a different aspect of a word. We can
interpret a sense vector by inspecting its (non-contextual, linear) projection
onto the output space, and intervene on these interpretable hooks to change the
model's behavior in predictable ways. We train a 170M-parameter Backpack
language model on OpenWebText, matching the loss of a GPT-2 small
(124Mparameter) Transformer. On lexical similarity evaluations, we find that
Backpack sense vectors outperform even a 6B-parameter Transformer LM's word
embeddings. Finally, we present simple algorithms that intervene on sense
vectors to perform controllable text generation and debiasing. For example, we
can edit the sense vocabulary to tend more towards a topic, or localize a
source of gender bias to a sense vector and globally suppress that sense.
Backpacks, a novel neural architecture, uses multiple sense vectors per word for context-dependent representation, offering interpretability and control with strong modeling performance.
neural architecture sense vectors context-dependent linear combination interpretability intervention language model OpenWebText GPT-2 Transformer lexical similarity controllable text generation debiasing