Controlling the Extraction of Memorized Data from Large Language Models
  via Prompt-Tuning
Large Language Models (LLMs) are known to memorize significant portions of
their training data. Parts of this memorized content have been shown to be
extractable by simply querying the model, which poses a privacy risk. We
present a novel approach which uses prompt-tuning to control the extraction
rates of memorized content in LLMs. We present two prompt training strategies
to increase and decrease extraction rates, which correspond to an attack and a
defense, respectively. We demonstrate the effectiveness of our techniques by
using models from the GPT-Neo family on a public benchmark. For the 1.3B
parameter GPT-Neo model, our attack yields a 9.3 percentage point increase in
extraction rate compared to our baseline. Our defense can be tuned to achieve
different privacy-utility trade-offs by a user-specified hyperparameter. We
achieve an extraction rate reduction of up to 97.7% relative to our baseline,
with a perplexity increase of 16.9%.
646ae91531fbfc5df82027a3
Mustafa Safa Ozdayi
646ae91531fbfc5df82027a4
Charith Peris
admin_assigned
2023-05-22T12:41:16.719Z
626029fdd47e3dbae32d89e2
https://cdn-avatars.huggingface.co/v1/production/uploads/1651094348689-626029fdd47e3dbae32d89e2.jpeg
Charith Peris
user
cperiz
646ae91531fbfc5df82027a5
Jack FitzGerald
admin_assigned
2023-05-22T12:41:58.568Z
6269d5cda6a7bba9e46dbea4
/avatars/6e2c4751ae8bad23fbb5af61ab535ee8.svg
Jack FitzGerald
user
jgmf
646ae91531fbfc5df82027a6
Christophe Dupuy
646ae91531fbfc5df82027a7
Jimit Majmudar
646ae91531fbfc5df82027a8
Haidar Khan
claimed_verified
2025-05-06T08:38:23.601Z
6556a6f90bd9dbcb06a257c4
/avatars/b315ba3a8e676ae7719e73a378cc0e07.svg
Haidar Khan
user
haidark1
646ae91531fbfc5df82027a9
Rahil Parikh
admin_assigned
2023-05-22T12:42:55.293Z
645ae69ce7e8a4b57bdc08f8
https://cdn-avatars.huggingface.co/v1/production/uploads/645ae69ce7e8a4b57bdc08f8/g26MrrfmzS8GxCuoccOsV.jpeg
Rahil Parikh
user
rhlprkh
646ae91531fbfc5df82027aa
Rahul Gupta
Controlling the Extraction of Memorized Data from Large Language Models
  via Prompt-Tuning
Large Language Models (LLMs) are known to memorize significant portions of
their training data. Parts of this memorized content have been shown to be
extractable by simply querying the model, which poses a privacy risk. We
present a novel approach which uses prompt-tuning to control the extraction
rates of memorized content in LLMs. We present two prompt training strategies
to increase and decrease extraction rates, which correspond to an attack and a
defense, respectively. We demonstrate the effectiveness of our techniques by
using models from the GPT-Neo family on a public benchmark. For the 1.3B
parameter GPT-Neo model, our attack yields a 9.3 percentage point increase in
extraction rate compared to our baseline. Our defense can be tuned to achieve
different privacy-utility trade-offs by a user-specified hyperparameter. We
achieve an extraction rate reduction of up to 97.7% relative to our baseline,
with a perplexity increase of 16.9%.
Prompt-tuning can control the extraction rates of memorized content in large language models, offering both attack and defense mechanisms to manage privacy risks.
prompt-tuning extraction rates memorized content LLMs GPT-Neo perplexity privacy-utility trade-offs