  While large language models (LLMs) are extensively used, there are raising
concerns regarding privacy, security, and copyright due to their opaque
training data, which brings the problem of detecting pre-training data on the
table. Current solutions to this problem leverage techniques explored in
machine learning privacy such as Membership Inference Attacks (MIAs), which
heavily depend on LLMs' capability of verbatim memorization. However, this
reliance presents challenges, especially given the vast amount of training data
and the restricted number of effective training epochs. In this paper, we
propose an adaptive pre-training data detection method which alleviates this
reliance and effectively amplify the identification. Our method adaptively
locates \textit{surprising tokens} of the input. A token is surprising to a LLM
if the prediction on the token is "certain but wrong", which refers to low
Shannon entropy of the probability distribution and low probability of the
ground truth token at the same time. By using the prediction probability of
surprising tokens to measure \textit{surprising}, the detection method is
achieved based on the simple hypothesis that seeing seen data is less
surprising for the model compared with seeing unseen data. The method can be
applied without any access to the the pre-training data corpus or additional
training like reference models. Our approach exhibits a consistent enhancement
compared to existing methods in diverse experiments conducted on various
benchmarks and models, achieving a maximum improvement of 29.5\%. We also
introduce a new benchmark Dolma-Book developed upon a novel framework, which
employs book data collected both before and after model training to provide
further evaluation.
