ConflictBank: A Benchmark for Evaluating the Influence of Knowledge
  Conflicts in LLM
Large language models (LLMs) have achieved impressive advancements across
numerous disciplines, yet the critical issue of knowledge conflicts, a major
source of hallucinations, has rarely been studied. Only a few research explored
the conflicts between the inherent knowledge of LLMs and the retrieved
contextual knowledge. However, a thorough assessment of knowledge conflict in
LLMs is still missing. Motivated by this research gap, we present ConflictBank,
the first comprehensive benchmark developed to systematically evaluate
knowledge conflicts from three aspects: (i) conflicts encountered in retrieved
knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the
interplay between these conflict forms. Our investigation delves into four
model families and twelve LLM instances, meticulously analyzing conflicts
stemming from misinformation, temporal discrepancies, and semantic divergences.
Based on our proposed novel construction framework, we create 7,453,853
claim-evidence pairs and 553,117 QA pairs. We present numerous findings on
model scale, conflict causes, and conflict types. We hope our ConflictBank
benchmark will help the community better understand model behavior in conflicts
and develop more reliable LLMs.
66c8481a2400073af3fbd0ae
Zhaochen Su
claimed_verified
2025-01-10T08:21:48.988Z
64264095ba51f8a2136946a0
https://cdn-avatars.huggingface.co/v1/production/uploads/64264095ba51f8a2136946a0/FR33boVpkDXcrvGMBmprF.jpeg
Zhaochen Su
user
Warrieryes
66c8481a2400073af3fbd0af
Jun Zhang
66c8481a2400073af3fbd0b0
Xiaoye Qu
claimed_verified
2024-10-04T08:46:39.836Z
64cb54da1af278541d663708
/avatars/c44507cc92bb2e83154bad31b90ce6dd.svg
Xiaoye Qu
user
Xiaoye08
66c8481a2400073af3fbd0b1
Tong Zhu
claimed_verified
2024-10-04T09:02:51.536Z
629454301ae2138079f7ff31
https://cdn-avatars.huggingface.co/v1/production/uploads/629454301ae2138079f7ff31/rVtbF-j06gDiYzomTeVTc.jpeg
Tong Zhu
user
Spico
66c8481a2400073af3fbd0b2
Yanshu Li
66c8481a2400073af3fbd0b3
Jiashuo Sun
claimed_verified
2025-05-14T07:37:52.927Z
64527f701a57e1179c1c3693
/avatars/25b2632d7aa9ce26d5d4924ecb00c4f4.svg
Jiashuo Sun
user
gasolsun
66c8481a2400073af3fbd0b4
Juntao Li
claimed_verified
2024-10-04T20:11:41.428Z
60d1d7f94672e8c13e7c2bce
/avatars/be3a3990492ef1c5f3ff33cc043ae166.svg
Juntao Li
user
ljtsuda
66c8481a2400073af3fbd0b5
Min Zhang
66c8481a2400073af3fbd0b6
Yu Cheng
claimed_verified
2024-10-09T16:26:33.596Z
67017abfe4d49b157ac534d9
/avatars/997e1b9f54b27a7728a9d4abfee4ba91.svg
Yu Cheng
user
ych133
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge
  Conflicts in LLM
Large language models (LLMs) have achieved impressive advancements across
numerous disciplines, yet the critical issue of knowledge conflicts, a major
source of hallucinations, has rarely been studied. Only a few research explored
the conflicts between the inherent knowledge of LLMs and the retrieved
contextual knowledge. However, a thorough assessment of knowledge conflict in
LLMs is still missing. Motivated by this research gap, we present ConflictBank,
the first comprehensive benchmark developed to systematically evaluate
knowledge conflicts from three aspects: (i) conflicts encountered in retrieved
knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the
interplay between these conflict forms. Our investigation delves into four
model families and twelve LLM instances, meticulously analyzing conflicts
stemming from misinformation, temporal discrepancies, and semantic divergences.
Based on our proposed novel construction framework, we create 7,453,853
claim-evidence pairs and 553,117 QA pairs. We present numerous findings on
model scale, conflict causes, and conflict types. We hope our ConflictBank
benchmark will help the community better understand model behavior in conflicts
and develop more reliable LLMs.
ConflictBank is a benchmark for evaluating knowledge conflicts in LLMs by assessing retrieved knowledge, model-encoded knowledge, and their interplay across different model families and causes like misinformation and temporal discrepancies.
large language models LLMs knowledge conflicts hallucinations retrieved knowledge encoded knowledge ConflictBank misinformation temporal discrepancies claim-evidence pairs QA pairs