Estimating the Hallucination Rate of Generative AI
This work is about estimating the hallucination rate for in-context learning
(ICL) with Generative AI. In ICL, a conditional generative model (CGM) is
prompted with a dataset and asked to make a prediction based on that dataset.
The Bayesian interpretation of ICL assumes that the CGM is calculating a
posterior predictive distribution over an unknown Bayesian model of a latent
parameter and data. With this perspective, we define a hallucination
as a generated prediction that has low-probability under the true latent
parameter. We develop a new method that takes an ICL problem -- that is, a CGM,
a dataset, and a prediction question -- and estimates the probability that a
CGM will generate a hallucination. Our method only requires generating queries
and responses from the model and evaluating its response log probability. We
empirically evaluate our method on synthetic regression and natural language
ICL tasks using large language models.
666c64b370f5a2cb4a5fd753
Andrew Jesson
admin_assigned
2024-06-14T15:45:25.211Z
666a65a66cf1faccbd4c85ea
/avatars/97262276275537ce3926efd93ac5a086.svg
andrew jesson
user
anndvision
666c64b370f5a2cb4a5fd754
Nicolas Beltran-Velez
666c64b370f5a2cb4a5fd755
Quentin Chu
admin_assigned
2024-06-14T15:45:38.744Z
65089ef1ce83a0c12a664e33
https://cdn-avatars.huggingface.co/v1/production/uploads/65089ef1ce83a0c12a664e33/zT6F1yndVKXueqYc-OebE.jpeg
Quentin Chu
user
qc2321
666c64b370f5a2cb4a5fd756
Sweta Karlekar
admin_assigned
2024-06-14T15:45:53.881Z
65e0dcaf35191b15a3873835
/avatars/65d44808417b133cf83216122a1b511e.svg
Sweta Karlekar
user
swkarlekar
666c64b370f5a2cb4a5fd757
Jannik Kossen
claimed_verified
2024-06-14T15:45:46.815Z
64df30f2e672e4d2c57049e8
/avatars/8c9503e43f403d655b11ea5c6e23ae79.svg
Jannik
user
jlko
666c64b370f5a2cb4a5fd758
Yarin Gal
666c64b370f5a2cb4a5fd759
John P. Cunningham
666c64b370f5a2cb4a5fd75a
David Blei
Estimating the Hallucination Rate of Generative AI
This work is about estimating the hallucination rate for in-context learning
(ICL) with Generative AI. In ICL, a conditional generative model (CGM) is
prompted with a dataset and asked to make a prediction based on that dataset.
The Bayesian interpretation of ICL assumes that the CGM is calculating a
posterior predictive distribution over an unknown Bayesian model of a latent
parameter and data. With this perspective, we define a hallucination
as a generated prediction that has low-probability under the true latent
parameter. We develop a new method that takes an ICL problem -- that is, a CGM,
a dataset, and a prediction question -- and estimates the probability that a
CGM will generate a hallucination. Our method only requires generating queries
and responses from the model and evaluating its response log probability. We
empirically evaluate our method on synthetic regression and natural language
ICL tasks using large language models.
A new method estimates the hallucination rate in in-context learning by assessing the log probability of model responses compared to the true latent parameter.
in-context learning (ICL) conditional generative model (CGM) Bayesian interpretation posterior predictive distribution latent parameter hallucination log probability