Detecting Pretraining Data from Large Language Models
Although large language models (LLMs) are widely deployed, the data used to
train them is rarely disclosed. Given the incredible scale of this data, up to
trillions of tokens, it is all but certain that it includes potentially
problematic text such as copyrighted materials, personally identifiable
information, and test data for widely reported reference benchmarks. However,
we currently have no way to know which data of these types is included or in
what proportions. In this paper, we study the pretraining data detection
problem: given a piece of text and black-box access to an LLM without knowing
the pretraining data, can we determine if the model was trained on the provided
text? To facilitate this study, we introduce a dynamic benchmark WIKIMIA that
uses data created before and after model training to support gold truth
detection. We also introduce a new detection method Min-K% Prob based on a
simple hypothesis: an unseen example is likely to contain a few outlier words
with low probabilities under the LLM, while a seen example is less likely to
have words with such low probabilities. Min-K% Prob can be applied without any
knowledge about the pretraining corpus or any additional training, departing
from previous detection methods that require training a reference model on data
that is similar to the pretraining data. Moreover, our experiments demonstrate
that Min-K% Prob achieves a 7.4% improvement on WIKIMIA over these previous
methods. We apply Min-K% Prob to two real-world scenarios, copyrighted book
detection, and contaminated downstream example detection, and find it a
consistently effective solution.
6539de890d111541df1f4145
Weijia Shi
admin_assigned
2023-10-26T08:35:07.794Z
6400f2ed568dbe30c9161e47
/avatars/c55938df5bce82b5d96e592a1ec36a8b.svg
Weijia Shi
user
swj0419
6539de890d111541df1f4146
Anirudh Ajith
admin_assigned
2023-10-26T08:35:16.200Z
6244346e799d7b023c675158
/avatars/e39ca0b6295b229bcef990d2cbe3a295.svg
Anirudh Ajith
user
anirudhajith
6539de890d111541df1f4147
Mengzhou Xia
admin_assigned
2023-10-26T08:35:23.435Z
6241126dedf27fda71c6b573
/avatars/ac106139c0e5967776f39e2755ca0792.svg
Mengzhou Xia
user
mengzhouxia
6539de890d111541df1f4148
Yangsibo Huang
admin_assigned
2023-10-26T08:35:48.652Z
5f32f7f77e58354338621556
/avatars/b2d702848d7b6c404c28b9959f050fdc.svg
Yangsibo Huang
user
hazelsuko
6539de890d111541df1f4149
Daogao Liu
6539de890d111541df1f414a
Terra Blevins
admin_assigned
2023-10-26T08:36:10.488Z
64528b183f80ad88c7791862
/avatars/f25635c484d86b23db42289fc29eb73a.svg
Terra Blevins
user
blvns
6539de890d111541df1f414b
Danqi Chen
claimed_verified
2023-10-26T11:36:51.728Z
653a4ca8669b77a9f8034ec0
/avatars/005ba9ead6b95b9ee8a31d050de77cee.svg
Danqi Chen
user
cdq10131
6539de890d111541df1f414c
Luke Zettlemoyer
Detecting Pretraining Data from Large Language Models
Although large language models (LLMs) are widely deployed, the data used to
train them is rarely disclosed. Given the incredible scale of this data, up to
trillions of tokens, it is all but certain that it includes potentially
problematic text such as copyrighted materials, personally identifiable
information, and test data for widely reported reference benchmarks. However,
we currently have no way to know which data of these types is included or in
what proportions. In this paper, we study the pretraining data detection
problem: given a piece of text and black-box access to an LLM without knowing
the pretraining data, can we determine if the model was trained on the provided
text? To facilitate this study, we introduce a dynamic benchmark WIKIMIA that
uses data created before and after model training to support gold truth
detection. We also introduce a new detection method Min-K% Prob based on a
simple hypothesis: an unseen example is likely to contain a few outlier words
with low probabilities under the LLM, while a seen example is less likely to
have words with such low probabilities. Min-K% Prob can be applied without any
knowledge about the pretraining corpus or any additional training, departing
from previous detection methods that require training a reference model on data
that is similar to the pretraining data. Moreover, our experiments demonstrate
that Min-K% Prob achieves a 7.4% improvement on WIKIMIA over these previous
methods. We apply Min-K% Prob to two real-world scenarios, copyrighted book
detection, and contaminated downstream example detection, and find it a
consistently effective solution.
Researchers propose Min-K% Prob, a method for detecting pretraining data in large language models without needing knowledge of the pretraining corpus, using a benchmark that supports gold truth detection.
large language models pretraining data detection black-box access WIKIMIA Min-K% Prob outlier words low probabilities