  Adversarial training (AT) is currently one of the most effective ways to
obtain the robustness of deep neural networks against adversarial attacks.
However, most AT methods suffer from robust overfitting, i.e., a significant
generalization gap in adversarial robustness between the training and testing
curves. In this paper, we first identify a connection between robust
overfitting and the excessive memorization of noisy labels in AT from a view of
gradient norm. As such label noise is mainly caused by a distribution mismatch
and improper label assignments, we are motivated to propose a label refinement
approach for AT. Specifically, our Self-Guided Label Refinement first
self-refines a more accurate and informative label distribution from
over-confident hard labels, and then it calibrates the training by dynamically
incorporating knowledge from self-distilled models into the current model and
thus requiring no external teachers. Empirical results demonstrate that our
method can simultaneously boost the standard accuracy and robust performance
across multiple benchmark datasets, attack types, and architectures. In
addition, we also provide a set of analyses from the perspectives of
information theory to dive into our method and suggest the importance of soft
labels for robust generalization.
