  Good models require good training data. For overparameterized deep models,
the causal relationship between training data and model predictions is
increasingly opaque and poorly understood. Influence analysis partially
demystifies training's underlying interactions by quantifying the amount each
training instance alters the final model. Measuring the training data's
influence exactly can be provably hard in the worst case; this has led to the
development and use of influence estimators, which only approximate the true
influence. This paper provides the first comprehensive survey of training data
influence analysis and estimation. We begin by formalizing the various, and in
places orthogonal, definitions of training data influence. We then organize
state-of-the-art influence analysis methods into a taxonomy; we describe each
of these methods in detail and compare their underlying assumptions, asymptotic
complexities, and overall strengths and weaknesses. Finally, we propose future
research directions to make influence analysis more useful in practice as well
as more theoretically and empirically sound. A curated, up-to-date list of
resources related to influence analysis is available at
https://github.com/ZaydH/influence_analysis_papers.
