Strong Membership Inference Attacks on Massive Datasets and (Moderately)
  Large Language Models
State-of-the-art membership inference attacks (MIAs) typically require
training many reference models, making it difficult to scale these attacks to
large pre-trained language models (LLMs). As a result, prior research has
either relied on weaker attacks that avoid training reference models (e.g.,
fine-tuning attacks), or on stronger attacks applied to small-scale models and
datasets. However, weaker attacks have been shown to be brittle - achieving
close-to-arbitrary success - and insights from strong attacks in simplified
settings do not translate to today's LLMs. These challenges have prompted an
important question: are the limitations observed in prior work due to attack
design choices, or are MIAs fundamentally ineffective on LLMs? We address this
question by scaling LiRA - one of the strongest MIAs - to GPT-2 architectures
ranging from 10M to 1B parameters, training reference models on over 20B tokens
from the C4 dataset. Our results advance the understanding of MIAs on LLMs in
three key ways: (1) strong MIAs can succeed on pre-trained LLMs; (2) their
effectiveness, however, remains limited (e.g., AUC<0.7) in practical settings;
and, (3) the relationship between MIA success and related privacy metrics is
not as straightforward as prior work has suggested.
6835727f9da2b91fb4e30473
Jamie Hayes
6835727f9da2b91fb4e30474
Ilia Shumailov
admin_assigned
2025-05-27T12:44:17.787Z
6475c2794766357252e69e9f
/avatars/757ed789423113369868e972f21ce559.svg
i
user
iliashum
6835727f9da2b91fb4e30475
Christopher A. Choquette-Choo
6835727f9da2b91fb4e30476
Matthew Jagielski
admin_assigned
2025-05-27T12:44:27.202Z
654a4a3b6a46bad8c0f2ab80
/avatars/08caf74525ea3e46fe58c20eb1f3d64d.svg
Matthew Jagielski
user
jagielski
6835727f9da2b91fb4e30477
George Kaissis
6835727f9da2b91fb4e30478
Katherine Lee
6835727f9da2b91fb4e30479
Milad Nasr
6835727f9da2b91fb4e3047a
Sahra Ghalebikesabi
admin_assigned
2025-05-27T12:44:52.007Z
633a01c6c0b7fd23493cb934
/avatars/8592c548b2f8324c43c5fed2599e6074.svg
Sahra Ghalebikesabi
user
sahra
6835727f9da2b91fb4e3047b
Niloofar Mireshghallah
6835727f9da2b91fb4e3047c
Meenatchi Sundaram Mutu Selva Annamalai
6835727f9da2b91fb4e3047d
Igor Shilov
admin_assigned
2025-05-27T12:45:06.168Z
6579c19a7a0f9b8a1abdccf1
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vhsOT4mneeJX0KtFi-uDb.jpeg
Igor Shilov
user
ffuuugor
6835727f9da2b91fb4e3047e
Matthieu Meeus
claimed_verified
2025-05-28T09:00:24.642Z
63d66b586f49aa82305e848d
https://cdn-avatars.huggingface.co/v1/production/uploads/63d66b586f49aa82305e848d/ZlN9lKBwSEsEKl6yqddZZ.jpeg
Matthieu Meeus
user
matthieumeeus97
6835727f9da2b91fb4e3047f
Yves-Alexandre de Montjoye
admin_assigned
2025-05-27T12:45:18.016Z
6694f6b3284bcf975861b818
/avatars/37cd86306142027663b6471b0033ef58.svg
Yves-Alexandre de Montjoye
user
yvesalexandre
6835727f9da2b91fb4e30480
Franziska Boenisch
6835727f9da2b91fb4e30481
Adam Dziedzic
6835727f9da2b91fb4e30482
A. Feder Cooper
extracted_confirmed
2025-05-27T20:00:14.988Z
663c3b587e7bc3d3e4a54ffb
/avatars/681abf7e4a85184667015cefefa226c6.svg
A. Feder Cooper
user
pasta41
Strong Membership Inference Attacks on Massive Datasets and (Moderately)
  Large Language Models
State-of-the-art membership inference attacks (MIAs) typically require
training many reference models, making it difficult to scale these attacks to
large pre-trained language models (LLMs). As a result, prior research has
either relied on weaker attacks that avoid training reference models (e.g.,
fine-tuning attacks), or on stronger attacks applied to small-scale models and
datasets. However, weaker attacks have been shown to be brittle - achieving
close-to-arbitrary success - and insights from strong attacks in simplified
settings do not translate to today's LLMs. These challenges have prompted an
important question: are the limitations observed in prior work due to attack
design choices, or are MIAs fundamentally ineffective on LLMs? We address this
question by scaling LiRA - one of the strongest MIAs - to GPT-2 architectures
ranging from 10M to 1B parameters, training reference models on over 20B tokens
from the C4 dataset. Our results advance the understanding of MIAs on LLMs in
three key ways: (1) strong MIAs can succeed on pre-trained LLMs; (2) their
effectiveness, however, remains limited (e.g., AUC<0.7) in practical settings;
and, (3) the relationship between MIA success and related privacy metrics is
not as straightforward as prior work has suggested.
Scaling LiRA membership inference attacks to large pre-trained language models shows that while these attacks can succeed, their effectiveness is limited and does not definitively correlate with privacy metrics.
membership inference attacks MIAs reference models fine-tuning attacks pre-trained language models LLMs LiRA GPT-2 tokens C4 dataset AUC privacy metrics