  Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks
that would trigger misclassification of DNNs but may be imperceptible to human
perception. Adversarial defense has been an important way to improve the
robustness of DNNs. Existing attack methods often construct adversarial
examples relying on some metrics like the $\ell_p$ distance to perturb samples.
However, these metrics can be insufficient to conduct adversarial attacks due
to their limited perturbations. In this paper, we propose a new internal
Wasserstein distance (IWD) to capture the semantic similarity of two samples,
and thus it helps to obtain larger perturbations than currently used metrics
such as the $\ell_p$ distance. We then apply the internal Wasserstein distance
to perform adversarial attack and defense. In particular, we develop a novel
attack method relying on IWD to calculate the similarities between an image and
its adversarial examples. In this way, we can generate diverse and semantically
similar adversarial examples that are more difficult to defend by existing
defense methods. Moreover, we devise a new defense method relying on IWD to
learn robust models against unseen adversarial examples. We provide both
thorough theoretical and empirical evidence to support our methods.
