LookAhead Tuning: Safer Language Models via Partial Answer Previews
Fine-tuning enables large language models (LLMs) to adapt to specific
domains, but often undermines their previously established safety alignment. To
mitigate the degradation of model safety during fine-tuning, we introduce
LookAhead Tuning, which comprises two simple, low-resource, and effective
data-driven methods that modify training data by previewing partial answer
prefixes. Both methods aim to preserve the model's inherent safety mechanisms
by minimizing perturbations to initial token distributions. Comprehensive
experiments demonstrate that LookAhead Tuning effectively maintains model
safety without sacrificing robust performance on downstream tasks. Our findings
position LookAhead Tuning as a reliable and efficient solution for the safe and
effective adaptation of LLMs. Code is released at
https://github.com/zjunlp/LookAheadTuning.
67e35da0b1b97cc3392024b1
Kangwei Liu
67e35da0b1b97cc3392024b2
Mengru Wang
67e35da0b1b97cc3392024b3
Yujie Luo
67e35da0b1b97cc3392024b4
Lin Yuan
67e35da0b1b97cc3392024b5
Mengshu Sun
67e35da0b1b97cc3392024b6
Ningyu Zhang
claimed_verified
2025-05-21T12:27:22.449Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
67e35da0b1b97cc3392024b7
Lei Liang
67e35da0b1b97cc3392024b8
Zhiqiang Zhang
67e35da0b1b97cc3392024b9
Jun Zhou
67e35da0b1b97cc3392024ba
Huajun Chen
LookAhead Tuning: Safer Language Models via Partial Answer Previews
Fine-tuning enables large language models (LLMs) to adapt to specific
domains, but often undermines their previously established safety alignment. To
mitigate the degradation of model safety during fine-tuning, we introduce
LookAhead Tuning, which comprises two simple, low-resource, and effective
data-driven methods that modify training data by previewing partial answer
prefixes. Both methods aim to preserve the model's inherent safety mechanisms
by minimizing perturbations to initial token distributions. Comprehensive
experiments demonstrate that LookAhead Tuning effectively maintains model
safety without sacrificing robust performance on downstream tasks. Our findings
position LookAhead Tuning as a reliable and efficient solution for the safe and
effective adaptation of LLMs. Code is released at
https://github.com/zjunlp/LookAheadTuning.
LookAhead Tuning, a data-driven approach, enhances the safety of large language models during fine-tuning by minimizing token distribution changes, while maintaining robust performance.
LookAhead Tuning large language models (LLMs) safety alignment fine-tuning token distribution safety mechanisms