  Given the complexity and lack of transparency in deep neural networks (DNNs),
extensive efforts have been made to make these systems more interpretable or
explain their behaviors in accessible terms. Unlike most reviews, which focus
on algorithmic and model-centric perspectives, this work takes a "data-centric"
view, examining how data collection, processing, and analysis contribute to
explainable AI (XAI). We categorize existing work into three categories subject
to their purposes: interpretations of deep models, referring to feature
attributions and reasoning processes that correlate data points with model
outputs; influences of training data, examining the impact of training data
nuances, such as data valuation and sample anomalies, on decision-making
processes; and insights of domain knowledge, discovering latent patterns and
fostering new knowledge from data and models to advance social values and
scientific discovery. Specifically, we distill XAI methodologies into data
mining operations on training and testing data across modalities, such as
images, text, and tabular data, as well as on training logs, checkpoints,
models and other DNN behavior descriptors. In this way, our study offers a
comprehensive, data-centric examination of XAI from a lens of data mining
methods and applications.
