  Large language models are trained on vast amounts of internet data, prompting
concerns and speculation that they have memorized public benchmarks. Going from
speculation to proof of contamination is challenging, as the pretraining data
used by proprietary models are often not publicly accessible. We show that it
is possible to provide provable guarantees of test set contamination in
language models without access to pretraining data or model weights. Our
approach leverages the fact that when there is no data contamination, all
orderings of an exchangeable benchmark should be equally likely. In contrast,
the tendency for language models to memorize example order means that a
contaminated language model will find certain canonical orderings to be much
more likely than others. Our test flags potential contamination whenever the
likelihood of a canonically ordered benchmark dataset is significantly higher
than the likelihood after shuffling the examples. We demonstrate that our
procedure is sensitive enough to reliably prove test set contamination in
challenging situations, including models as small as 1.4 billion parameters, on
small test sets of only 1000 examples, and datasets that appear only a few
times in the pretraining corpus. Using our test, we audit five popular publicly
accessible language models for test set contamination and find little evidence
for pervasive contamination.
