Be like a Goldfish, Don't Memorize! Mitigating Memorization in
  Generative LLMs
Large language models can memorize and repeat their training data, causing
privacy and copyright risks. To mitigate memorization, we introduce a subtle
modification to the next-token training objective that we call the goldfish
loss. During training, a randomly sampled subset of tokens are excluded from
the loss computation. These dropped tokens are not memorized by the model,
which prevents verbatim reproduction of a complete chain of tokens from the
training set. We run extensive experiments training billion-scale Llama-2
models, both pre-trained and trained from scratch, and demonstrate significant
reductions in extractable memorization with little to no impact on downstream
benchmarks.
6670da301b7f558a303a9061
Abhimanyu Hans
claimed_verified
2024-06-19T07:39:21.524Z
6489c06c08b6a836a6dbd6aa
/avatars/6ed1928992fc9d24bad0f12ebd2d84fe.svg
Abhimanyu Hans
user
ahans1
6670da301b7f558a303a9062
Yuxin Wen
6670da301b7f558a303a9063
Neel Jain
claimed_verified
2024-06-18T07:18:05.872Z
63e2b1ec282ee5f9624cfbcb
https://cdn-avatars.huggingface.co/v1/production/uploads/63e2b1ec282ee5f9624cfbcb/4SVTp93cvRevacoJgiXzS.jpeg
Neel Jain
user
nsjain
6670da301b7f558a303a9064
John Kirchenbauer
6670da301b7f558a303a9065
Hamid Kazemi
6670da301b7f558a303a9066
Prajwal Singhania
6670da301b7f558a303a9067
Siddharth Singh
6670da301b7f558a303a9068
Gowthami Somepalli
6670da301b7f558a303a9069
Jonas Geiping
6670da301b7f558a303a906a
Abhinav Bhatele
6670da301b7f558a303a906b
Tom Goldstein
Be like a Goldfish, Don't Memorize! Mitigating Memorization in
  Generative LLMs
Large language models can memorize and repeat their training data, causing
privacy and copyright risks. To mitigate memorization, we introduce a subtle
modification to the next-token training objective that we call the goldfish
loss. During training, a randomly sampled subset of tokens are excluded from
the loss computation. These dropped tokens are not memorized by the model,
which prevents verbatim reproduction of a complete chain of tokens from the
training set. We run extensive experiments training billion-scale Llama-2
models, both pre-trained and trained from scratch, and demonstrate significant
reductions in extractable memorization with little to no impact on downstream
benchmarks.
goldfish loss next-token training objective training objective randomly sampled subset loss computation dropped tokens model memorization verbatim reproduction training set perplexity downstream benchmarks billion-scale Llama-2 models pre-trained models trained from scratch