When to Speak, When to Abstain: Contrastive Decoding with Abstention
Large Language Models (LLMs) demonstrate exceptional performance across
diverse tasks by leveraging both pre-trained knowledge (i.e., parametric
knowledge) and external knowledge (i.e., contextual knowledge). While
substantial efforts have been made to leverage both forms of knowledge,
scenarios in which the model lacks any relevant knowledge remain underexplored.
Such limitations can result in issues like hallucination, causing reduced
reliability and potential risks in high-stakes applications. To address such
limitations, this paper extends the task scope to encompass cases where the
user's request cannot be fulfilled due to the lack of relevant knowledge. To
this end, we introduce Contrastive Decoding with Abstention (CDA), a
training-free decoding method that empowers LLMs to generate responses when
relevant knowledge is available and to abstain otherwise. CDA evaluates the
relevance of each knowledge for a given query, adaptively determining which
knowledge to prioritize or which to completely ignore. Extensive experiments
with four LLMs on three question-answering datasets demonstrate that CDA can
effectively perform accurate generation and abstention simultaneously. These
findings highlight CDA's potential to broaden the applicability of LLMs,
enhancing reliability and preserving user trust.
6762627c1b66b28261b4bf71
Hyuhng Joon Kim
claimed_verified
2024-12-18T09:51:18.410Z
6077e727048132eef27894b0
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/R49vipboidpLkGra_h1z5.png
Hyuhng Joon Kim
user
heyjoonkim
6762627c1b66b28261b4bf72
Youna Kim
6762627c1b66b28261b4bf73
Sang-goo Lee
6762627c1b66b28261b4bf74
Taeuk Kim
When to Speak, When to Abstain: Contrastive Decoding with Abstention
Large Language Models (LLMs) demonstrate exceptional performance across
diverse tasks by leveraging both pre-trained knowledge (i.e., parametric
knowledge) and external knowledge (i.e., contextual knowledge). While
substantial efforts have been made to leverage both forms of knowledge,
scenarios in which the model lacks any relevant knowledge remain underexplored.
Such limitations can result in issues like hallucination, causing reduced
reliability and potential risks in high-stakes applications. To address such
limitations, this paper extends the task scope to encompass cases where the
user's request cannot be fulfilled due to the lack of relevant knowledge. To
this end, we introduce Contrastive Decoding with Abstention (CDA), a
training-free decoding method that empowers LLMs to generate responses when
relevant knowledge is available and to abstain otherwise. CDA evaluates the
relevance of each knowledge for a given query, adaptively determining which
knowledge to prioritize or which to completely ignore. Extensive experiments
with four LLMs on three question-answering datasets demonstrate that CDA can
effectively perform accurate generation and abstention simultaneously. These
findings highlight CDA's potential to broaden the applicability of LLMs,
enhancing reliability and preserving user trust.
Contrastive Decoding with Abstention (CDA) enhances Large Language Models by enabling them to generate accurate responses or abstain when relevant knowledge is unavailable, thereby improving reliability and user trust.
Large Language Models parametric knowledge contextual knowledge hallucination Contrastive Decoding with Abstention CDA knowledge relevance question-answering datasets