  Adversarial robustness has become a central goal in deep learning, both in
the theory and the practice. However, successful methods to improve the
adversarial robustness (such as adversarial training) greatly hurt
generalization performance on the unperturbed data. This could have a major
impact on how the adversarial robustness affects real world systems (i.e. many
may opt to forego robustness if it can improve accuracy on the unperturbed
data). We propose Interpolated Adversarial Training, which employs recently
proposed interpolation based training methods in the framework of adversarial
training. On CIFAR-10, adversarial training increases the standard test error
(when there is no adversary) from 4.43% to 12.32%, whereas with our
Interpolated adversarial training we retain the adversarial robustness while
achieving a standard test error of only 6.45%. With our technique, the relative
increase in the standard error for the robust model is reduced from 178.1% to
just 45.5%. Moreover, we provide mathematical analysis of Interpolated
Adversarial Training to confirm its efficiencies and demonstrate its advantages
in terms of robustness and generalization.
