On the Robustness of Dataset Inference
Machine learning (ML) models are costly to train as they can require a
significant amount of data, computational resources and technical expertise.
Thus, they constitute valuable intellectual property that needs protection from
adversaries wanting to steal them. Ownership verification techniques allow the
victims of model stealing attacks to demonstrate that a suspect model was in
fact stolen from theirs.
  Although a number of ownership verification techniques based on watermarking
or fingerprinting have been proposed, most of them fall short either in terms
of security guarantees (well-equipped adversaries can evade verification) or
computational cost. A fingerprinting technique, Dataset Inference (DI), has
been shown to offer better robustness and efficiency than prior methods.
  The authors of DI provided a correctness proof for linear (suspect) models.
However, in a subspace of the same setting, we prove that DI suffers from high
false positives (FPs) -- it can incorrectly identify an independent model
trained with non-overlapping data from the same distribution as stolen. We
further prove that DI also triggers FPs in realistic, non-linear suspect
models. We then confirm empirically that DI in the black-box setting leads to
FPs, with high confidence.
  Second, we show that DI also suffers from false negatives (FNs) -- an
adversary can fool DI (at the cost of incurring some accuracy loss) by
regularising a stolen model's decision boundaries using adversarial training,
thereby leading to an FN. To this end, we demonstrate that black-box DI fails
to identify a model adversarially trained from a stolen dataset -- the setting
where DI is the hardest to evade.
  Finally, we discuss the implications of our findings, the viability of
fingerprinting-based ownership verification in general, and suggest directions
for future work.