TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long
  Documents
Large language models (LLMs) have attracted huge interest in practical
applications given their increasingly accurate responses and coherent reasoning
abilities. Given their nature as black-boxes using complex reasoning processes
on their inputs, it is inevitable that the demand for scalable and faithful
explanations for LLMs' generated content will continue to grow. There have been
major developments in the explainability of neural network models over the past
decade. Among them, post-hoc explainability methods, especially Shapley values,
have proven effective for interpreting deep learning models. However, there are
major challenges in scaling up Shapley values for LLMs, particularly when
dealing with long input contexts containing thousands of tokens and
autoregressively generated output sequences. Furthermore, it is often unclear
how to effectively utilize generated explanations to improve the performance of
LLMs. In this paper, we introduce TextGenSHAP, an efficient post-hoc
explanation method incorporating LM-specific techniques. We demonstrate that
this leads to significant increases in speed compared to conventional Shapley
value computations, reducing processing times from hours to minutes for
token-level explanations, and to just seconds for document-level explanations.
In addition, we demonstrate how real-time Shapley values can be utilized in two
important scenarios, providing better understanding of long-document question
answering by localizing important words and sentences; and improving existing
document retrieval systems through enhancing the accuracy of selected passages
and ultimately the final responses.
656e96b62cd7a3e348fff821
James Enouen
admin_assigned
2023-12-05T12:24:50.130Z
64fd49f70e486522f85f0bc2
/avatars/71ac3b7b5111aafb465062bb408145cd.svg
James Enouen
user
4Z0E50Xg5pQ
656e96b62cd7a3e348fff822
Hootan Nakhost
656e96b62cd7a3e348fff823
Sayna Ebrahimi
admin_assigned
2023-12-05T12:25:03.197Z
647e51337f9ad5e44baafe68
/avatars/264715eaeec7a28a9fea89f12732d8a3.svg
Sayna Ebrahimi
user
SaynaEbrahimi
656e96b62cd7a3e348fff824
Sercan O Arik
656e96b62cd7a3e348fff825
Yan Liu
656e96b62cd7a3e348fff826
Tomas Pfister
TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long
  Documents
Large language models (LLMs) have attracted huge interest in practical
applications given their increasingly accurate responses and coherent reasoning
abilities. Given their nature as black-boxes using complex reasoning processes
on their inputs, it is inevitable that the demand for scalable and faithful
explanations for LLMs' generated content will continue to grow. There have been
major developments in the explainability of neural network models over the past
decade. Among them, post-hoc explainability methods, especially Shapley values,
have proven effective for interpreting deep learning models. However, there are
major challenges in scaling up Shapley values for LLMs, particularly when
dealing with long input contexts containing thousands of tokens and
autoregressively generated output sequences. Furthermore, it is often unclear
how to effectively utilize generated explanations to improve the performance of
LLMs. In this paper, we introduce TextGenSHAP, an efficient post-hoc
explanation method incorporating LM-specific techniques. We demonstrate that
this leads to significant increases in speed compared to conventional Shapley
value computations, reducing processing times from hours to minutes for
token-level explanations, and to just seconds for document-level explanations.
In addition, we demonstrate how real-time Shapley values can be utilized in two
important scenarios, providing better understanding of long-document question
answering by localizing important words and sentences; and improving existing
document retrieval systems through enhancing the accuracy of selected passages
and ultimately the final responses.
TextGenSHAP is an efficient post-hoc explanation method for LLMs using Shapley values, significantly reducing computation time and improving applications like document question answering and retrieval.
large language models (LLMs) post-hoc explainability Shapley values token-level explanations document-level explanations real-time Shapley values long-document question answering document retrieval