  Intentionally crafted adversarial samples have effectively exploited
weaknesses in deep neural networks. A standard method in adversarial robustness
assumes a framework to defend against samples crafted by minimally perturbing a
sample such that its corresponding model output changes. These sensitivity
attacks exploit the model's sensitivity toward task-irrelevant features.
Another form of adversarial sample can be crafted via invariance attacks, which
exploit the model underestimating the importance of relevant features. Previous
literature has indicated a tradeoff in defending against both attack types
within a strictly L_p bounded defense. To promote robustness toward both types
of attacks beyond Euclidean distance metrics, we use metric learning to frame
adversarial regularization as an optimal transport problem. Our preliminary
results indicate that regularizing over invariant perturbations in our
framework improves both invariant and sensitivity defense.
