Steering Llama 2 via Contrastive Activation Addition
We introduce Contrastive Activation Addition (CAA), an innovative method for
steering language models by modifying activations during their forward passes.
CAA computes ``steering vectors'' by averaging the difference in residual
stream activations between pairs of positive and negative examples of a
particular behavior such as factual versus hallucinatory responses. During
inference, these steering vectors are added at all token positions after the
user's prompt with either a positive or negative coefficient, allowing precise
control over the degree of the targeted behavior. We evaluate CAA's
effectiveness on Llama 2 Chat using both multiple-choice behavioral question
datasets and open-ended generation tasks. We demonstrate that CAA significantly
alters model behavior, outperforms traditional methods like finetuning and
few-shot prompting, and minimally reduces capabilities. Moreover, by employing
various activation space interpretation methods, we gain deeper insights into
CAA's mechanisms. CAA both accurately steers model outputs and also sheds light
on how high-level concepts are represented in Large Language Models (LLMs).
657918cb7f5da1deb6a2e9b3
Nina Rimsky
extracted_confirmed
2023-12-13T02:39:58.373Z
64b80fee5c4af63e10825753
/avatars/68af35718e61097bd1f901d85121d96c.svg
Nina Panickssery
user
ninapanickssery
657918cb7f5da1deb6a2e9b4
Nick Gabrieli
657918cb7f5da1deb6a2e9b5
Julian Schulz
657918cb7f5da1deb6a2e9b6
Meg Tong
admin_assigned
2023-12-13T10:04:50.660Z
63272a638624baac667c8bdb
https://cdn-avatars.huggingface.co/v1/production/uploads/63272a638624baac667c8bdb/ylZ-FNT9PLhn8sBCD1wQm.png
Meg Tong
user
meg-tong
657918cb7f5da1deb6a2e9b7
Evan Hubinger
657918cb7f5da1deb6a2e9b8
Alexander Matt Turner
extracted_confirmed
2023-12-20T00:18:40.223Z
63d03d73623a3d1d11708951
/avatars/254d82bb0172c3a0ee90c85379fd19bd.svg
Alex Turner
user
TurnTrout
Steering Llama 2 via Contrastive Activation Addition
We introduce Contrastive Activation Addition (CAA), an innovative method for
steering language models by modifying activations during their forward passes.
CAA computes ``steering vectors'' by averaging the difference in residual
stream activations between pairs of positive and negative examples of a
particular behavior such as factual versus hallucinatory responses. During
inference, these steering vectors are added at all token positions after the
user's prompt with either a positive or negative coefficient, allowing precise
control over the degree of the targeted behavior. We evaluate CAA's
effectiveness on Llama 2 Chat using both multiple-choice behavioral question
datasets and open-ended generation tasks. We demonstrate that CAA significantly
alters model behavior, outperforms traditional methods like finetuning and
few-shot prompting, and minimally reduces capabilities. Moreover, by employing
various activation space interpretation methods, we gain deeper insights into
CAA's mechanisms. CAA both accurately steers model outputs and also sheds light
on how high-level concepts are represented in Large Language Models (LLMs).
Contrastive Activation Addition (CAA) modifies model activations to steer language model behavior with high precision and insight into high-level concept representation.
Contrastive Activation Addition CAA steering vectors residual stream token positions behavioral question datasets open-ended generation large language models LLMs