Blackbox Model Provenance via Palimpsestic Membership Inference
Rohith Kuditipudi, Jing Huang, Sally Zhu, Percy Liang, Christopher Potts, Diyi Yang
Suppose Alice trains an open-weight language model, and subsequently Bob uses a blackbox derivative of Alice's model to produce text. Can Alice prove that Bob is using her model, either by querying Bob's derivative model (query setting) or from the text alone (sample setting)? We formulate this question as an independence testing problem---in which the null hypothesis is that Bob's model is independent of Alice's randomized training run---and investigate it through the lens of \textit{palimpsestic memorization} in language models: models are more likely to memorize data seen later in training, so we can test whether Bob's model derives from Alice's using test statistics that capture correlation between the output of Bob's model and the ordering of examples in Alice's training run. So long as Alice has randomly shuffled her training data, any significant correlation amounts to exactly quantifiable statistical evidence against the null hypothesis, regardless of the composition of Alice's training data. We develop tests for both the query and sample settings and empirically validate the power of our tests using the Pythia and OLMo model families, as well as small-scale models trained on TinyStories. In the query setting, we query Bob's model on Alice's training data and measure the correlation of its log-likelihood with the ordering of data. We show that this test is robust to common post-training practices (e.g., supervised fine-tuning, preference optimization, model souping). In the sample setting, we match spans of Bob's text against Alice's training examples and correlate the likelihood of a match with the ordering of training examples. We show this test reliably attributes text to models given a few thousand tokens. Our work offers a novel framework for provenance verification of open-weight language models, enabling accountability and protection for models.