When to Trust Context: Self-Reflective Debates for Context Reliability
Large language models frequently encounter conflicts between their parametric
knowledge and contextual input, often resulting in factual inconsistencies or
hallucinations. We propose Self-Reflective Debate for Contextual Reliability
(SR-DCR), a lightweight framework that integrates token-level self-confidence
with an asymmetric multi-agent debate to adjudicate such conflicts. A critic,
deprived of context, challenges a defender who argues from the given passage; a
judge model evaluates the debate and determines the context's reliability. The
final answer is selected by combining the verdict with model confidence.
Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently
enhances robustness to misleading context while maintaining accuracy on
trustworthy inputs, outperforming both classical debate and confidence-only
baselines with minimal computational overhead. The code is available at
https://github.com/smiles724/Self-Reflective-Debates.
684aff253b733ba333686e6e
Zeqi Zhou
684aff253b733ba333686e6f
Fang Wu
extracted_confirmed
2025-07-22T18:10:16.998Z
675e0d5cdd3e9eeed6954f5a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7oMEoBmaFiCR9K2q9Z_7q.png
Fang Wu
user
fangwu97
684aff253b733ba333686e70
Shayan Talaei
684aff253b733ba333686e71
Haokai Zhao
684aff253b733ba333686e72
Cheng Meixin
684aff253b733ba333686e73
Tinson Xu
684aff253b733ba333686e74
Amin Saberi
684aff253b733ba333686e75
Yejin Choi
When to Trust Context: Self-Reflective Debates for Context Reliability
Large language models frequently encounter conflicts between their parametric
knowledge and contextual input, often resulting in factual inconsistencies or
hallucinations. We propose Self-Reflective Debate for Contextual Reliability
(SR-DCR), a lightweight framework that integrates token-level self-confidence
with an asymmetric multi-agent debate to adjudicate such conflicts. A critic,
deprived of context, challenges a defender who argues from the given passage; a
judge model evaluates the debate and determines the context's reliability. The
final answer is selected by combining the verdict with model confidence.
Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently
enhances robustness to misleading context while maintaining accuracy on
trustworthy inputs, outperforming both classical debate and confidence-only
baselines with minimal computational overhead. The code is available at
https://github.com/smiles724/Self-Reflective-Debates.
A lightweight framework integrating token-level self-confidence and an asymmetric debate between agents enhances the robustness of large language models to contextual inconsistencies with minimal computational cost.
token-level self-confidence asymmetric multi-agent debate critic defender judge model context reliability factual inconsistencies hallucinations ClashEval benchmark