  Recent work has found that adversarially-robust deep networks used for image
classification are more interpretable: their feature attributions tend to be
sharper, and are more concentrated on the objects associated with the image's
ground-truth class. We show that smooth decision boundaries play an important
role in this enhanced interpretability, as the model's input gradients around
data points will more closely align with boundaries' normal vectors when they
are smooth. Thus, because robust models have smoother boundaries, the results
of gradient-based attribution methods, like Integrated Gradients and DeepLift,
will capture more accurate information about nearby decision boundaries. This
understanding of robust interpretability leads to our second contribution:
\emph{boundary attributions}, which aggregate information about the normal
vectors of local decision boundaries to explain a classification outcome. We
show that by leveraging the key factors underpinning robust interpretability,
boundary attributions produce sharper, more concentrated visual explanations --
even on non-robust models. Any example implementation can be found at
\url{https://github.com/zifanw/boundary}.
