  With the widespread applications of deep convolutional neural networks
(DCNNs), it becomes increasingly important for DCNNs not only to make accurate
predictions but also to explain how they make their decisions. In this work, we
propose a CHannel-wise disentangled InterPretation (CHIP) model to give the
visual interpretation to the predictions of DCNNs. The proposed model distills
the class-discriminative importance of channels in networks by utilizing the
sparse regularization. Here, we first introduce the network perturbation
technique to learn the model. The proposed model is capable to not only distill
the global perspective knowledge from networks but also present the
class-discriminative visual interpretation for specific predictions of
networks. It is noteworthy that the proposed model is able to interpret
different layers of networks without re-training. By combining the distilled
interpretation knowledge in different layers, we further propose the Refined
CHIP visual interpretation that is both high-resolution and
class-discriminative. Experimental results on the standard dataset demonstrate
that the proposed model provides promising visual interpretation for the
predictions of networks in image classification task compared with existing
visual interpretation methods. Besides, the proposed method outperforms related
approaches in the application of ILSVRC 2015 weakly-supervised localization
task.
