UnUnlearning: Unlearning is not sufficient for content regulation in
  advanced generative AI
Exact unlearning was first introduced as a privacy mechanism that allowed a
user to retract their data from machine learning models on request. Shortly
after, inexact schemes were proposed to mitigate the impractical costs
associated with exact unlearning. More recently unlearning is often discussed
as an approach for removal of impermissible knowledge i.e. knowledge that the
model should not possess such as unlicensed copyrighted, inaccurate, or
malicious information. The promise is that if the model does not have a certain
malicious capability, then it cannot be used for the associated malicious
purpose. In this paper we revisit the paradigm in which unlearning is used for
in Large Language Models (LLMs) and highlight an underlying inconsistency
arising from in-context learning. Unlearning can be an effective control
mechanism for the training phase, yet it does not prevent the model from
performing an impermissible act during inference. We introduce a concept of
ununlearning, where unlearned knowledge gets reintroduced in-context,
effectively rendering the model capable of behaving as if it knows the
forgotten knowledge. As a result, we argue that content filtering for
impermissible knowledge will be required and even exact unlearning schemes are
not enough for effective content regulation. We discuss feasibility of
ununlearning for modern LLMs and examine broader implications.
6683b18f5f3099b62dd0555f
Ilia Shumailov
6683b18f5f3099b62dd05560
Jamie Hayes
6683b18f5f3099b62dd05561
Eleni Triantafillou
6683b18f5f3099b62dd05562
Guillermo Ortiz-Jimenez
6683b18f5f3099b62dd05563
Nicolas Papernot
6683b18f5f3099b62dd05564
Matthew Jagielski
admin_assigned
2024-07-03T15:09:39.176Z
654a4a3b6a46bad8c0f2ab80
/avatars/08caf74525ea3e46fe58c20eb1f3d64d.svg
Matthew Jagielski
user
jagielski
6683b18f5f3099b62dd05565
Itay Yona
admin_assigned
2024-07-03T15:09:33.536Z
62697edaa6a7bba9e46ae4ad
/avatars/3009bf769a09d946447a0e8c833a04f3.svg
Itay Yona
user
tux
6683b18f5f3099b62dd05566
Heidi Howard
6683b18f5f3099b62dd05567
Eugene Bagdasaryan
admin_assigned
2024-07-03T15:09:19.593Z
61ae7dc0f3f39c685adeb738
https://cdn-avatars.huggingface.co/v1/production/uploads/1638978358427-61ae7dc0f3f39c685adeb738.jpeg
Eugene Bagdasaryan
user
ebagdasa
UnUnlearning: Unlearning is not sufficient for content regulation in
  advanced generative AI
Exact unlearning was first introduced as a privacy mechanism that allowed a
user to retract their data from machine learning models on request. Shortly
after, inexact schemes were proposed to mitigate the impractical costs
associated with exact unlearning. More recently unlearning is often discussed
as an approach for removal of impermissible knowledge i.e. knowledge that the
model should not possess such as unlicensed copyrighted, inaccurate, or
malicious information. The promise is that if the model does not have a certain
malicious capability, then it cannot be used for the associated malicious
purpose. In this paper we revisit the paradigm in which unlearning is used for
in Large Language Models (LLMs) and highlight an underlying inconsistency
arising from in-context learning. Unlearning can be an effective control
mechanism for the training phase, yet it does not prevent the model from
performing an impermissible act during inference. We introduce a concept of
ununlearning, where unlearned knowledge gets reintroduced in-context,
effectively rendering the model capable of behaving as if it knows the
forgotten knowledge. As a result, we argue that content filtering for
impermissible knowledge will be required and even exact unlearning schemes are
not enough for effective content regulation. We discuss feasibility of
ununlearning for modern LLMs and examine broader implications.
Unlearning in Large Language Models is found to be ineffective due to in-context learning, implying that additional content filtering mechanisms are necessary.
unlearning Large Language Models in-context learning ununlearning content filtering