ContextCite: Attributing Model Generation to Context
How do language models use information provided as context when generating a
response? Can we infer whether a particular generated statement is actually
grounded in the context, a misinterpretation, or fabricated? To help answer
these questions, we introduce the problem of context attribution: pinpointing
the parts of the context (if any) that led a model to generate a particular
statement. We then present ContextCite, a simple and scalable method for
context attribution that can be applied on top of any existing language model.
Finally, we showcase the utility of ContextCite through three applications: (1)
helping verify generated statements (2) improving response quality by pruning
the context and (3) detecting poisoning attacks. We provide code for
ContextCite at https://github.com/MadryLab/context-cite.
66d802899fcb6b494f44536f
Benjamin Cohen-Wang
admin_assigned
2024-09-04T09:52:18.420Z
639aaf82a4c528850bba2bfe
https://cdn-avatars.huggingface.co/v1/production/uploads/639aaf82a4c528850bba2bfe/nn23r8bsNiOJzVUxAPfo7.png
Benjamin Cohen-Wang
user
bencw
66d802899fcb6b494f445370
Harshay Shah
claimed_verified
2024-09-04T08:04:28.641Z
64b1a4f64dd3e24895daa236
https://cdn-avatars.huggingface.co/v1/production/uploads/64b1a4f64dd3e24895daa236/lzlZ4DBOw-YVspNttAEY3.jpeg
Harshay Shah
user
harshay
66d802899fcb6b494f445371
Kristian Georgiev
admin_assigned
2024-09-04T09:52:23.838Z
63b98550efe99543b34cd33d
https://cdn-avatars.huggingface.co/v1/production/uploads/63b98550efe99543b34cd33d/lsfiSVfQeKOypMh66c8pT.jpeg
Kristian Georgiev
user
krisgrg
66d802899fcb6b494f445372
Aleksander Madry
ContextCite: Attributing Model Generation to Context
How do language models use information provided as context when generating a
response? Can we infer whether a particular generated statement is actually
grounded in the context, a misinterpretation, or fabricated? To help answer
these questions, we introduce the problem of context attribution: pinpointing
the parts of the context (if any) that led a model to generate a particular
statement. We then present ContextCite, a simple and scalable method for
context attribution that can be applied on top of any existing language model.
Finally, we showcase the utility of ContextCite through three applications: (1)
helping verify generated statements (2) improving response quality by pruning
the context and (3) detecting poisoning attacks. We provide code for
ContextCite at https://github.com/MadryLab/context-cite.
context attribution ContextCite