  It is well-known that deep neural networks (DNNs) are susceptible to
adversarial attacks, exposing a severe fragility of deep learning systems. As
the result, adversarial training (AT) method, by incorporating adversarial
examples during training, represents a natural and effective approach to
strengthen the robustness of a DNN-based classifier. However, most AT-based
methods, notably PGD-AT and TRADES, typically seek a pointwise adversary that
generates the worst-case adversarial example by independently perturbing each
data sample, as a way to "probe" the vulnerability of the classifier. Arguably,
there are unexplored benefits in considering such adversarial effects from an
entire distribution. To this end, this paper presents a unified framework that
connects Wasserstein distributional robustness with current state-of-the-art AT
methods. We introduce a new Wasserstein cost function and a new series of risk
functions, with which we show that standard AT methods are special cases of
their counterparts in our framework. This connection leads to an intuitive
relaxation and generalization of existing AT methods and facilitates the
development of a new family of distributional robustness AT-based algorithms.
Extensive experiments show that our distributional robustness AT algorithms
robustify further their standard AT counterparts in various settings.
