  Interpreting the decision logic behind effective deep convolutional neural
networks (CNN) on images complements the success of deep learning models.
However, the existing methods can only interpret some specific decision logic
on individual or a small number of images. To facilitate human
understandability and generalization ability, it is important to develop
representative interpretations that interpret common decision logics of a CNN
on a large group of similar images, which reveal the common semantics data
contributes to many closely related predictions. In this paper, we develop a
novel unsupervised approach to produce a highly representative interpretation
for a large number of similar images. We formulate the problem of finding
representative interpretations as a co-clustering problem, and convert it into
a submodular cost submodular cover problem based on a sample of the linear
decision boundaries of a CNN. We also present a visualization and similarity
ranking method. Our extensive experiments demonstrate the excellent performance
of our method.
