  The success of convolutional neural networks (CNNs) in various applications
is accompanied by a significant increase in computation and parameter storage
costs. Recent efforts to reduce these overheads involve pruning and compressing
the weights of various layers while at the same time aiming to not sacrifice
performance. In this paper, we propose a novel criterion for CNN pruning
inspired by neural network interpretability: The most relevant units, i.e.
weights or filters, are automatically found using their relevance scores
obtained from concepts of explainable AI (XAI). By exploring this idea, we
connect the lines of interpretability and model compression research. We show
that our proposed method can efficiently prune CNN models in transfer-learning
setups in which networks pre-trained on large corpora are adapted to
specialized tasks. The method is evaluated on a broad range of computer vision
datasets. Notably, our novel criterion is not only competitive or better
compared to state-of-the-art pruning criteria when successive retraining is
performed, but clearly outperforms these previous criteria in the
resource-constrained application scenario in which the data of the task to be
transferred to is very scarce and one chooses to refrain from fine-tuning. Our
method is able to compress the model iteratively while maintaining or even
improving accuracy. At the same time, it has a computational cost in the order
of gradient computation and is comparatively simple to apply without the need
for tuning hyperparameters for pruning.
