SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large
  Language Models
Safety-aligned language models often exhibit fragile and imbalanced safety
mechanisms, increasing the likelihood of generating unsafe content. In
addition, incorporating new knowledge through editing techniques to language
models can further compromise safety. To address these issues, we propose
SafeInfer, a context-adaptive, decoding-time safety alignment strategy for
generating safe responses to user queries. SafeInfer comprises two phases: the
safety amplification phase, which employs safe demonstration examples to adjust
the model's hidden states and increase the likelihood of safer outputs, and the
safety-guided decoding phase, which influences token selection based on
safety-optimized distributions, ensuring the generated content complies with
ethical guidelines. Further, we present HarmEval, a novel benchmark for
extensive safety evaluations, designed to address potential misuse scenarios in
accordance with the policies of leading AI tech giants.
667322720c3379a2a4d9df9f
Somnath Banerjee
claimed_verified
2024-06-20T12:42:53.587Z
62659abbe25d2b490db94d35
https://cdn-avatars.huggingface.co/v1/production/uploads/1650829776213-62659abbe25d2b490db94d35.jpeg
Somnath Banerjee
user
leowin
667322720c3379a2a4d9dfa0
Soham Tripathy
admin_assigned
2024-06-19T21:09:27.765Z
62a2e5442dac39c21736146e
/avatars/c09bfc7ccc8370b0524c889c26eef489.svg
Soham Tripathy
user
soham17
667322720c3379a2a4d9dfa1
Sayan Layek
admin_assigned
2024-06-19T21:09:50.381Z
653fc666625ef3d61e012fae
/avatars/a222546e8378543cf5c2f7c552414143.svg
Sayan Layek
user
caprion
667322720c3379a2a4d9dfa2
Shanu Kumar
claimed_verified
2024-10-30T09:56:24.820Z
63183c3ed78318df4abe92d8
/avatars/2821d66e9f88a8c5f35d55ec92ebd06e.svg
Shanu Kumar
user
sshanukr
667322720c3379a2a4d9dfa3
Animesh Mukherjee
admin_assigned
2024-06-19T21:10:00.413Z
5ebd450338bcf80db421091e
/avatars/13a1945aa90a7521c413ae0eb7b3228a.svg
Animesh Mukherjee
user
animeshm
667322720c3379a2a4d9dfa4
Rima Hazra
admin_assigned
2024-06-19T21:10:05.866Z
64be339512afb2f11951b217
https://cdn-avatars.huggingface.co/v1/production/uploads/64be339512afb2f11951b217/B-O3HH0lG0BOdmEE4sTAV.png
RIMA HAZRA
user
rimahazra
SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large
  Language Models
Safety-aligned language models often exhibit fragile and imbalanced safety
mechanisms, increasing the likelihood of generating unsafe content. In
addition, incorporating new knowledge through editing techniques to language
models can further compromise safety. To address these issues, we propose
SafeInfer, a context-adaptive, decoding-time safety alignment strategy for
generating safe responses to user queries. SafeInfer comprises two phases: the
safety amplification phase, which employs safe demonstration examples to adjust
the model's hidden states and increase the likelihood of safer outputs, and the
safety-guided decoding phase, which influences token selection based on
safety-optimized distributions, ensuring the generated content complies with
ethical guidelines. Further, we present HarmEval, a novel benchmark for
extensive safety evaluations, designed to address potential misuse scenarios in
accordance with the policies of leading AI tech giants.
SafeInfer enhances language models' safety by adjusting hidden states and guiding token selection, using a benchmark to ensure ethical compliance.
SafeInfer context-adaptive safety alignment safety amplification hidden states safety-guided decoding token selection HarmEval safety evaluations ethical guidelines