  Recent studies on the adversarial vulnerability of neural networks have shown
that models trained to be more robust to adversarial attacks exhibit more
interpretable saliency maps than their non-robust counterparts. We aim to
quantify this behavior by considering the alignment between input image and
saliency map. We hypothesize that as the distance to the decision boundary
grows,so does the alignment. This connection is strictly true in the case of
linear models. We confirm these theoretical findings with experiments based on
models trained with a local Lipschitz regularization and identify where the
non-linear nature of neural networks weakens the relation.
