  We introduce an analytical framework to quantify the changes in a machine
learning algorithm's output distribution following the inclusion of a few data
points in its training set, a notion we define as leave-one-out
distinguishability (LOOD). This is key to measuring data **memorization** and
information **leakage** as well as the **influence** of training data points in
machine learning. We illustrate how our method broadens and refines existing
empirical measures of memorization and privacy risks associated with training
data. We use Gaussian processes to model the randomness of machine learning
algorithms, and validate LOOD with extensive empirical analysis of leakage
using membership inference attacks. Our analytical framework enables us to
investigate the causes of leakage and where the leakage is high. For example,
we analyze the influence of activation functions, on data memorization.
Additionally, our method allows us to identify queries that disclose the most
information about the training data in the leave-one-out setting. We illustrate
how optimal queries can be used for accurate **reconstruction** of training
data.
