LLMs Know More Than They Show: On the Intrinsic Representation of LLM
  Hallucinations
Large language models (LLMs) often produce errors, including factual
inaccuracies, biases, and reasoning failures, collectively referred to as
"hallucinations". Recent studies have demonstrated that LLMs' internal states
encode information regarding the truthfulness of their outputs, and that this
information can be utilized to detect errors. In this work, we show that the
internal representations of LLMs encode much more information about
truthfulness than previously recognized. We first discover that the
truthfulness information is concentrated in specific tokens, and leveraging
this property significantly enhances error detection performance. Yet, we show
that such error detectors fail to generalize across datasets, implying that --
contrary to prior claims -- truthfulness encoding is not universal but rather
multifaceted. Next, we show that internal representations can also be used for
predicting the types of errors the model is likely to make, facilitating the
development of tailored mitigation strategies. Lastly, we reveal a discrepancy
between LLMs' internal encoding and external behavior: they may encode the
correct answer, yet consistently generate an incorrect one. Taken together,
these insights deepen our understanding of LLM errors from the model's internal
perspective, which can guide future research on enhancing error analysis and
mitigation.
6703267ec265fe2565c140da
Hadas Orgad
admin_assigned
2024-10-08T09:09:44.396Z
60f82853c53e95176a7c6d45
/avatars/16f6ea944a014af6ebe60499f3460784.svg
Hadas Orgad
user
hadasor
6703267ec265fe2565c140db
Michael Toker
admin_assigned
2024-10-08T09:10:10.493Z
61c865e3d3702a3bbf50bc04
https://cdn-avatars.huggingface.co/v1/production/uploads/61c865e3d3702a3bbf50bc04/eGgcyeUnOHz0hyap5vhvr.jpeg
Michael Toker
user
tokeron
6703267ec265fe2565c140dc
Zorik Gekhman
6703267ec265fe2565c140dd
Roi Reichart
6703267ec265fe2565c140de
Idan Szpektor
6703267ec265fe2565c140df
Hadas Kotek
6703267ec265fe2565c140e0
Yonatan Belinkov
admin_assigned
2024-10-08T09:10:41.670Z
614c57f1ee44bcfe57b366d6
/avatars/186a9aed84681246f48ed2a012c50def.svg
Yonatan Belinkov
user
belinkov
LLMs Know More Than They Show: On the Intrinsic Representation of LLM
  Hallucinations
Large language models (LLMs) often produce errors, including factual
inaccuracies, biases, and reasoning failures, collectively referred to as
"hallucinations". Recent studies have demonstrated that LLMs' internal states
encode information regarding the truthfulness of their outputs, and that this
information can be utilized to detect errors. In this work, we show that the
internal representations of LLMs encode much more information about
truthfulness than previously recognized. We first discover that the
truthfulness information is concentrated in specific tokens, and leveraging
this property significantly enhances error detection performance. Yet, we show
that such error detectors fail to generalize across datasets, implying that --
contrary to prior claims -- truthfulness encoding is not universal but rather
multifaceted. Next, we show that internal representations can also be used for
predicting the types of errors the model is likely to make, facilitating the
development of tailored mitigation strategies. Lastly, we reveal a discrepancy
between LLMs' internal encoding and external behavior: they may encode the
correct answer, yet consistently generate an incorrect one. Taken together,
these insights deepen our understanding of LLM errors from the model's internal
perspective, which can guide future research on enhancing error analysis and
mitigation.
LLMs internally encode detailed truthfulness information about their outputs, which can be used to detect and predict errors but shows variability across datasets and does not always align with the final output.
LLMs hallucinations internal states truthfulness information error detection generalization error types error mitigation internal encoding external behavior