  Adversarial training has been widely explored for mitigating attacks against
deep models. However, most existing works are still trapped in the dilemma
between higher accuracy and stronger robustness since they tend to fit a model
towards robust features (not easily tampered with by adversaries) while
ignoring those non-robust but highly predictive features. To achieve a better
robustness-accuracy trade-off, we propose the Vanilla Feature Distillation
Adversarial Training (VFD-Adv), which conducts knowledge distillation from a
pre-trained model (optimized towards high accuracy) to guide adversarial
training towards higher accuracy, i.e., preserving those non-robust but
predictive features. More specifically, both adversarial examples and their
clean counterparts are forced to be aligned in the feature space by distilling
predictive representations from the pre-trained/clean model, while previous
works barely utilize predictive features from clean models. Therefore, the
adversarial training model is updated towards maximally preserving the accuracy
as gaining robustness. A key advantage of our method is that it can be
universally adapted to and boost existing works. Exhaustive experiments on
various datasets, classification models, and adversarial training algorithms
demonstrate the effectiveness of our proposed method.
