The Surprising Effectiveness of Membership Inference with Simple N-Gram
  Coverage
Membership inference attacks serves as useful tool for fair use of language
models, such as detecting potential copyright infringement and auditing data
leakage. However, many current state-of-the-art attacks require access to
models' hidden states or probability distribution, which prevents investigation
into more widely-used, API-access only models like GPT-4. In this work, we
introduce N-Gram Coverage Attack, a membership inference attack that relies
solely on text outputs from the target model, enabling attacks on completely
black-box models. We leverage the observation that models are more likely to
memorize and subsequently generate text patterns that were commonly observed in
their training data. Specifically, to make a prediction on a candidate member,
N-Gram Coverage Attack first obtains multiple model generations conditioned on
a prefix of the candidate. It then uses n-gram overlap metrics to compute and
aggregate the similarities of these outputs with the ground truth suffix; high
similarities indicate likely membership. We first demonstrate on a diverse set
of existing benchmarks that N-Gram Coverage Attack outperforms other black-box
methods while also impressively achieving comparable or even better performance
to state-of-the-art white-box attacks - despite having access to only text
outputs. Interestingly, we find that the success rate of our method scales with
the attack compute budget - as we increase the number of sequences generated
from the target model conditioned on the prefix, attack performance tends to
improve. Having verified the accuracy of our method, we use it to investigate
previously unstudied closed OpenAI models on multiple domains. We find that
more recent models, such as GPT-4o, exhibit increased robustness to membership
inference, suggesting an evolving trend toward improved privacy protections.
689e84dba4caabb4320e5cbe
Skyler Hallinan
claimed_verified
2025-08-18T06:58:09.814Z
637d38fcb8e573d75bedc033
https://cdn-avatars.huggingface.co/v1/production/uploads/637d38fcb8e573d75bedc033/MbzfbMBGN2ajH7oZtitcB.png
Skyler Hallinan
user
hallisky
689e84dba4caabb4320e5cbf
Jaehun Jung
689e84dba4caabb4320e5cc0
Melanie Sclar
689e84dba4caabb4320e5cc1
Ximing Lu
689e84dba4caabb4320e5cc2
Abhilasha Ravichander
689e84dba4caabb4320e5cc3
Sahana Ramnath
689e84dba4caabb4320e5cc4
Yejin Choi
689e84dba4caabb4320e5cc5
Sai Praneeth Karimireddy
689e84dba4caabb4320e5cc6
Niloofar Mireshghallah
689e84dba4caabb4320e5cc7
Xiang Ren
The Surprising Effectiveness of Membership Inference with Simple N-Gram
  Coverage
Membership inference attacks serves as useful tool for fair use of language
models, such as detecting potential copyright infringement and auditing data
leakage. However, many current state-of-the-art attacks require access to
models' hidden states or probability distribution, which prevents investigation
into more widely-used, API-access only models like GPT-4. In this work, we
introduce N-Gram Coverage Attack, a membership inference attack that relies
solely on text outputs from the target model, enabling attacks on completely
black-box models. We leverage the observation that models are more likely to
memorize and subsequently generate text patterns that were commonly observed in
their training data. Specifically, to make a prediction on a candidate member,
N-Gram Coverage Attack first obtains multiple model generations conditioned on
a prefix of the candidate. It then uses n-gram overlap metrics to compute and
aggregate the similarities of these outputs with the ground truth suffix; high
similarities indicate likely membership. We first demonstrate on a diverse set
of existing benchmarks that N-Gram Coverage Attack outperforms other black-box
methods while also impressively achieving comparable or even better performance
to state-of-the-art white-box attacks - despite having access to only text
outputs. Interestingly, we find that the success rate of our method scales with
the attack compute budget - as we increase the number of sequences generated
from the target model conditioned on the prefix, attack performance tends to
improve. Having verified the accuracy of our method, we use it to investigate
previously unstudied closed OpenAI models on multiple domains. We find that
more recent models, such as GPT-4o, exhibit increased robustness to membership
inference, suggesting an evolving trend toward improved privacy protections.
N-Gram Coverage Attack is a black-box membership inference method that uses text outputs to detect data membership in language models, outperforming other black-box methods and achieving comparable results to white-box attacks.
membership inference attacks N-Gram Coverage Attack black-box models n-gram overlap metrics text outputs language models GPT-4 privacy protections