DyePack: Provably Flagging Test Set Contamination in LLMs Using
  Backdoors
Open benchmarks are essential for evaluating and advancing large language
models, offering reproducibility and transparency. However, their accessibility
makes them likely targets of test set contamination. In this work, we introduce
DyePack, a framework that leverages backdoor attacks to identify models that
used benchmark test sets during training, without requiring access to the loss,
logits, or any internal details of the model. Like how banks mix dye packs with
their money to mark robbers, DyePack mixes backdoor samples with the test data
to flag models that trained on it. We propose a principled design incorporating
multiple backdoors with stochastic targets, enabling exact false positive rate
(FPR) computation when flagging every model. This provably prevents false
accusations while providing strong evidence for every detected case of
contamination. We evaluate DyePack on five models across three datasets,
covering both multiple-choice and open-ended generation tasks. For
multiple-choice questions, it successfully detects all contaminated models with
guaranteed FPRs as low as 0.000073% on MMLU-Pro and 0.000017% on Big-Bench-Hard
using eight backdoors. For open-ended generation tasks, it generalizes well and
identifies all contaminated models on Alpaca with a guaranteed false positive
rate of just 0.127% using six backdoors.
6839f87c49d173e7b23f220b
Yize Cheng
extracted_confirmed
2025-05-30T18:27:47.059Z
66fa2c61c25c3fcb32f9f131
/avatars/05387c30f2d1803fa0a5b176c3706772.svg
Yize Cheng
user
yizecheng
6839f87c49d173e7b23f220c
Wenxiao Wang
extracted_confirmed
2025-05-30T18:28:18.975Z
659dc02d72238596c24d49f5
/avatars/d4600d23ccc72f296fab7f626d5895e7.svg
Wenxiao Wang
user
wangwenxiao
6839f87c49d173e7b23f220d
Mazda Moayeri
extracted_confirmed
2025-05-30T20:01:34.913Z
63449874ee1504dbcd59af3d
/avatars/57a805d82d16de9544c98585bd7a3e55.svg
MazdaM
user
mmoayeri
6839f87c49d173e7b23f220e
Soheil Feizi
DyePack: Provably Flagging Test Set Contamination in LLMs Using
  Backdoors
Open benchmarks are essential for evaluating and advancing large language
models, offering reproducibility and transparency. However, their accessibility
makes them likely targets of test set contamination. In this work, we introduce
DyePack, a framework that leverages backdoor attacks to identify models that
used benchmark test sets during training, without requiring access to the loss,
logits, or any internal details of the model. Like how banks mix dye packs with
their money to mark robbers, DyePack mixes backdoor samples with the test data
to flag models that trained on it. We propose a principled design incorporating
multiple backdoors with stochastic targets, enabling exact false positive rate
(FPR) computation when flagging every model. This provably prevents false
accusations while providing strong evidence for every detected case of
contamination. We evaluate DyePack on five models across three datasets,
covering both multiple-choice and open-ended generation tasks. For
multiple-choice questions, it successfully detects all contaminated models with
guaranteed FPRs as low as 0.000073% on MMLU-Pro and 0.000017% on Big-Bench-Hard
using eight backdoors. For open-ended generation tasks, it generalizes well and
identifies all contaminated models on Alpaca with a guaranteed false positive
rate of just 0.127% using six backdoors.
DyePack, a framework using backdoor attacks, identifies models that leveraged benchmark test sets during training by introducing benign backdoor samples, ensuring precise false positive rates while preventing false accusations.
backdoor attacks test set contamination false positive rate FPR DyePack multiple backdoors stochastic targets MMLU-Pro Big-Bench-Hard Alpaca multiple-choice questions open-ended generation tasks