  Model interpretability is a requirement in many applications in which crucial
decisions are made by users relying on a model's outputs. The recent movement
for "algorithmic fairness" also stipulates explainability, and therefore
interpretability of learning models. And yet the most successful contemporary
Machine Learning approaches, the Deep Neural Networks, produce models that are
highly non-interpretable. We attempt to address this challenge by proposing a
technique called CNN-INTE to interpret deep Convolutional Neural Networks (CNN)
via meta-learning. In this work, we interpret a specific hidden layer of the
deep CNN model on the MNIST image dataset. We use a clustering algorithm in a
two-level structure to find the meta-level training data and Random Forest as
base learning algorithms to generate the meta-level test data. The
interpretation results are displayed visually via diagrams, which clearly
indicates how a specific test instance is classified. Our method achieves
global interpretation for all the test instances without sacrificing the
accuracy obtained by the original deep CNN model. This means our model is
faithful to the deep CNN model, which leads to reliable interpretations.
