Be like a Goldfish, Don't Memorize! Mitigating Memorization in
  Generative LLMs
Large language models can memorize and repeat their training data, causing
privacy and copyright risks. To mitigate memorization, we introduce a subtle
modification to the next-token training objective that we call the goldfish
loss. During training, a randomly sampled subset of tokens are excluded from
the loss computation. These dropped tokens are not memorized by the model,
which prevents verbatim reproduction of a complete chain of tokens from the
training set. We run extensive experiments training billion-scale Llama-2
models, both pre-trained and trained from scratch, and demonstrate significant
reductions in extractable memorization with little to no impact on downstream
benchmarks.
6670da301b7f558a303a9061
Abhimanyu Hans
claimed_verified
2024-06-19T07:39:21.524Z
6489c06c08b6a836a6dbd6aa
/avatars/6ed1928992fc9d24bad0f12ebd2d84fe.svg
Abhimanyu Hans
user
ahans1
6670da301b7f558a303a9062
Yuxin Wen
6670da301b7f558a303a9063
Neel Jain
claimed_verified
2024-06-18T07:18:05.872Z
63e2b1ec282ee5f9624cfbcb
https://cdn-avatars.huggingface.co/v1/production/uploads/63e2b1ec282ee5f9624cfbcb/4SVTp93cvRevacoJgiXzS.jpeg
Neel Jain
user
nsjain
6670da301b7f558a303a9064
John Kirchenbauer
6670da301b7f558a303a9065
Hamid Kazemi
6670da301b7f558a303a9066
Prajwal Singhania
6670da301b7f558a303a9067
Siddharth Singh
6670da301b7f558a303a9068
Gowthami Somepalli
6670da301b7f558a303a9069
Jonas Geiping
claimed_verified
2025-06-27T08:58:56.888Z
63d86dbf3130cadcaf8bdd11
/avatars/29d79a0c6dcec01111ef192fecd0fa7a.svg
Jonas Geiping
user
JonasGeiping
6670da301b7f558a303a906a
Abhinav Bhatele
6670da301b7f558a303a906b
Tom Goldstein
Be like a Goldfish, Don't Memorize! Mitigating Memorization in
  Generative LLMs
Large language models can memorize and repeat their training data, causing
privacy and copyright risks. To mitigate memorization, we introduce a subtle
modification to the next-token training objective that we call the goldfish
loss. During training, a randomly sampled subset of tokens are excluded from
the loss computation. These dropped tokens are not memorized by the model,
which prevents verbatim reproduction of a complete chain of tokens from the
training set. We run extensive experiments training billion-scale Llama-2
models, both pre-trained and trained from scratch, and demonstrate significant
reductions in extractable memorization with little to no impact on downstream
benchmarks.
A goldfish loss, which excludes a random subset of tokens from the training loss, reduces memorization in large language models with minimal impact on performance.
goldfish loss next-token training objective large language models memorization privacy copyright extractable memorization Llama-2 models