Model Internals-based Answer Attribution for Trustworthy
  Retrieval-Augmented Generation
Ensuring the verifiability of model answers is a fundamental challenge for
retrieval-augmented generation (RAG) in the question answering (QA) domain.
Recently, self-citation prompting was proposed to make large language models
(LLMs) generate citations to supporting documents along with their answers.
However, self-citing LLMs often struggle to match the required format, refer to
non-existent sources, and fail to faithfully reflect LLMs' context usage
throughout the generation. In this work, we present MIRAGE --Model
Internals-based RAG Explanations -- a plug-and-play approach using model
internals for faithful answer attribution in RAG applications. MIRAGE detects
context-sensitive answer tokens and pairs them with retrieved documents
contributing to their prediction via saliency methods. We evaluate our proposed
approach on a multilingual extractive QA dataset, finding high agreement with
human answer attribution. On open-ended QA, MIRAGE achieves citation quality
and efficiency comparable to self-citation while also allowing for a
finer-grained control of attribution parameters. Our qualitative evaluation
highlights the faithfulness of MIRAGE's attributions and underscores the
promising application of model internals for RAG answer attribution.
667586138794bbd33fde01f3
Jirui Qi
claimed_verified
2024-06-21T15:00:34.930Z
645a415077d040cfd64e036e
https://cdn-avatars.huggingface.co/v1/production/uploads/645a415077d040cfd64e036e/2k6QsstgWC1H4oRMlzcc5.png
Jirui Qi
user
JRQi
667586138794bbd33fde01f4
Gabriele Sarti
claimed_verified
2024-06-21T14:20:43.979Z
5e7749883d77a72421292d07
https://cdn-avatars.huggingface.co/v1/production/uploads/5e7749883d77a72421292d07/M4AmBReZk_otxCIG3o0bL.jpeg
Gabriele Sarti
user
gsarti
667586138794bbd33fde01f5
Raquel Fernández
extracted_confirmed
2025-02-07T16:42:58.263Z
66055fcc196d49210e4f8ad9
/avatars/a6d3d10929fcf99e58391a55b5250bd4.svg
Raquel Fernández
user
rfernandez
667586138794bbd33fde01f6
Arianna Bisazza
Model Internals-based Answer Attribution for Trustworthy
  Retrieval-Augmented Generation
Ensuring the verifiability of model answers is a fundamental challenge for
retrieval-augmented generation (RAG) in the question answering (QA) domain.
Recently, self-citation prompting was proposed to make large language models
(LLMs) generate citations to supporting documents along with their answers.
However, self-citing LLMs often struggle to match the required format, refer to
non-existent sources, and fail to faithfully reflect LLMs' context usage
throughout the generation. In this work, we present MIRAGE --Model
Internals-based RAG Explanations -- a plug-and-play approach using model
internals for faithful answer attribution in RAG applications. MIRAGE detects
context-sensitive answer tokens and pairs them with retrieved documents
contributing to their prediction via saliency methods. We evaluate our proposed
approach on a multilingual extractive QA dataset, finding high agreement with
human answer attribution. On open-ended QA, MIRAGE achieves citation quality
and efficiency comparable to self-citation while also allowing for a
finer-grained control of attribution parameters. Our qualitative evaluation
highlights the faithfulness of MIRAGE's attributions and underscores the
promising application of model internals for RAG answer attribution.
MIRAGE, a plug-and-play method using model internals, enhances retrieval-augmented generation by accurately attributing answer tokens to relevant documents, achieving high agreement with human attributions and competitive citation quality.
retrieval-augmented generation RAG question answering QA self-citation prompting large language models LLMs MIRAGE model internals extractive QA open-ended QA saliency methods answer attribution