Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
In human-written articles, we often leverage the subtleties of text style,
such as bold and italics, to guide the attention of readers. These textual
emphases are vital for the readers to grasp the conveyed information. When
interacting with large language models (LLMs), we have a similar need -
steering the model to pay closer attention to user-specified information, e.g.,
an instruction. Existing methods, however, are constrained to process plain
text and do not support such a mechanism. This motivates us to introduce PASTA
- Post-hoc Attention STeering Approach, a method that allows LLMs to read text
with user-specified emphasis marks. To this end, PASTA identifies a small
subset of attention heads and applies precise attention reweighting on them,
directing the model attention to user-specified parts. Like prompting, PASTA is
applied at inference time and does not require changing any model parameters.
Experiments demonstrate that PASTA can substantially enhance an LLM's ability
to follow user instructions or integrate new knowledge from user inputs,
leading to a significant performance improvement on a variety of tasks, e.g.,
an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly
available at https://github.com/QingruZhang/PASTA .
6549e8b0137b7246f0479cf1
Qingru Zhang
admin_assigned
2023-11-07T15:43:26.696Z
64a76bf4a352114d1cbe0c3a
/avatars/966a556d86760dd9fe244cacbed9415d.svg
Qingru Zhang
user
zhang4562
6549e8b0137b7246f0479cf2
Chandan Singh
claimed_verified
2023-11-07T17:12:43.236Z
62b0bee117fa24819dd0999c
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0bee117fa24819dd0999c/eJ7ryFQ3_ZbqV29qf4KyM.jpeg
Chandan Singh
user
csinva
6549e8b0137b7246f0479cf3
Liyuan Liu
admin_assigned
2023-11-07T16:12:47.916Z
6476372871d420a1f6be3650
/avatars/ca3492d3967e06a7785457dd0ba2582c.svg
Liyuan Liu
user
LiyuanLucasLiu
6549e8b0137b7246f0479cf4
Xiaodong Liu
admin_assigned
2023-11-07T16:12:54.817Z
64b6007dfc5a8ae2a7e761ca
/avatars/7d59d9e19ea536ce388a83764e48867d.svg
xiaodong liu
user
ArivnLiu
6549e8b0137b7246f0479cf5
Bin Yu
extracted_confirmed
2024-02-21T14:57:41.867Z
64679498374fe5728d4044d5
/avatars/78516e1c9532900cce19f81483855db8.svg
Bin Yu
user
binyu000
6549e8b0137b7246f0479cf6
Jianfeng Gao
admin_assigned
2023-11-07T16:13:01.145Z
641904caf9d6f1d772ec7af7
/avatars/4a63eac71eb30f70b1a0e9d4708f26c1.svg
Jianfeng Gao
user
wyngjf
6549e8b0137b7246f0479cf7
Tuo Zhao
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
In human-written articles, we often leverage the subtleties of text style,
such as bold and italics, to guide the attention of readers. These textual
emphases are vital for the readers to grasp the conveyed information. When
interacting with large language models (LLMs), we have a similar need -
steering the model to pay closer attention to user-specified information, e.g.,
an instruction. Existing methods, however, are constrained to process plain
text and do not support such a mechanism. This motivates us to introduce PASTA
- Post-hoc Attention STeering Approach, a method that allows LLMs to read text
with user-specified emphasis marks. To this end, PASTA identifies a small
subset of attention heads and applies precise attention reweighting on them,
directing the model attention to user-specified parts. Like prompting, PASTA is
applied at inference time and does not require changing any model parameters.
Experiments demonstrate that PASTA can substantially enhance an LLM's ability
to follow user instructions or integrate new knowledge from user inputs,
leading to a significant performance improvement on a variety of tasks, e.g.,
an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly
available at https://github.com/QingruZhang/PASTA .
PASTA enhances LLM performance by directing model attention to user-specified parts through post-hoc attention reweighting without altering model parameters.
LLMs attention heads attention reweighting prompting inference time