  In this paper, we propose a self-interpretable model SITE with
transformation-equivariant interpretations. We focus on the robustness and
self-consistency of the interpretations of geometric transformations. Apart
from the transformation equivariance, as a self-interpretable model, SITE has
comparable expressive power as the benchmark black-box classifiers, while being
able to present faithful and robust interpretations with high quality. It is
worth noticing that although applied in most of the CNN visualization methods,
the bilinear upsampling approximation is a rough approximation, which can only
provide interpretations in the form of heatmaps (instead of pixel-wise). It
remains an open question whether such interpretations can be direct to the
input space (as shown in the MNIST experiments). Besides, we consider the
translation and rotation transformations in our model. In future work, we will
explore the robust interpretations under more complex transformations such as
scaling and distortion. Moreover, we clarify that SITE is not limited to
geometric transformation (that we used in the computer vision domain), and will
explore SITEin other domains in future work.
