  Adversarial robust models have been shown to learn more robust and
interpretable features than standard trained models. As shown in
[\cite{tsipras2018robustness}], such robust models inherit useful interpretable
properties where the gradient aligns perceptually well with images, and adding
a large targeted adversarial perturbation leads to an image resembling the
target class. We perform experiments to show that interpretable and
perceptually aligned gradients are present even in models that do not show high
robustness to adversarial attacks. Specifically, we perform adversarial
training with attack for different max-perturbation bound. Adversarial training
with low max-perturbation bound results in models that have interpretable
features with only slight drop in performance over clean samples. In this
paper, we leverage models with interpretable perceptually-aligned features and
show that adversarial training with low max-perturbation bound can improve the
performance of models for zero-shot and weakly supervised localization tasks.
