Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably
include contradictory factual information from sources of varying reliability.
In this paper, we propose measuring an LLM property called trusted source
alignment (TSA): the model's propensity to align with content produced by
trusted publishers in the face of uncertainty or controversy. We present
FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
articles. We describe a simple protocol for evaluating TSA and offer a detailed
analysis of design considerations including response extraction, claim
contextualization, and bias in prompt formulation. Applying the protocol to
PaLM-2, we find that as we scale up the model size, the model performance on
FactCheckQA improves from near-random to up to 80% balanced accuracy in
aligning with trusted sources.
6552ec71b2d47c6675c991c3
Vasilisa Bashlovkina
6552ec71b2d47c6675c991c4
Zhaobin Kuang
6552ec71b2d47c6675c991c5
Riley Matthews
6552ec71b2d47c6675c991c6
Edward Clifford
6552ec71b2d47c6675c991c7
Yennie Jun
admin_assigned
2023-11-14T11:06:54.076Z
60b1100b5b0f651abe90a474
/avatars/64ac849a0022aa4b4debf7a2d1629a09.svg
yennie jun
user
yenniejun
6552ec71b2d47c6675c991c8
William W. Cohen
6552ec71b2d47c6675c991c9
Simon Baumgartner
claimed_verified
2023-11-27T08:43:24.824Z
6560f697e0a7720b6ae377bc
https://cdn-avatars.huggingface.co/v1/production/uploads/6560f697e0a7720b6ae377bc/8b-p-lQ7KV_VygBJ6Vf3_.jpeg
Simon Baumgartner
user
sens3
Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably
include contradictory factual information from sources of varying reliability.
In this paper, we propose measuring an LLM property called trusted source
alignment (TSA): the model's propensity to align with content produced by
trusted publishers in the face of uncertainty or controversy. We present
FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
articles. We describe a simple protocol for evaluating TSA and offer a detailed
analysis of design considerations including response extraction, claim
contextualization, and bias in prompt formulation. Applying the protocol to
PaLM-2, we find that as we scale up the model size, the model performance on
FactCheckQA improves from near-random to up to 80% balanced accuracy in
aligning with trusted sources.
trusted source alignment (TSA) fact checking articles response extraction claim contextualization bias in prompt formulation