  While many have shown how Large Language Models (LLMs) can be applied to a
diverse set of tasks, the critical issues of data contamination and
memorization are often glossed over. In this work, we address this concern for
tabular data. Starting with simple qualitative tests for whether an LLM knows
the names and values of features, we introduce a variety of different
techniques to assess the degrees of contamination, including statistical tests
for conditional distribution modeling and four tests that identify
memorization. Our investigation reveals that LLMs are pre-trained on many
popular tabular datasets. This exposure can lead to invalid performance
evaluation on downstream tasks because the LLMs have, in effect, been fit to
the test set. Interestingly, we also identify a regime where the language model
reproduces important statistics of the data, but fails to reproduce the dataset
verbatim. On these datasets, although seen during training, good performance on
downstream tasks might not be due to overfitting. Our findings underscore the
need for ensuring data integrity in machine learning tasks with LLMs. To
facilitate future research, we release an open-source tool that can perform
various tests for memorization
\url{https://github.com/interpretml/LLM-Tabular-Memorization-Checker}.
