  Adversarial training (AT) is an effective defense method against
gradient-based attacks to enhance the robustness of neural networks. Among
them, single-step AT has emerged as a hotspot topic due to its simplicity and
efficiency, requiring only one gradient propagation in generating adversarial
examples. Nonetheless, the problem of catastrophic overfitting (CO) that causes
training collapse remains poorly understood, and there exists a gap between the
robust accuracy achieved through single- and multi-step AT. In this paper, we
present a surprising finding that the taxonomy of adversarial examples reveals
the truth of CO. Based on this conclusion, we propose taxonomy driven fast
adversarial training (TDAT) which jointly optimizes learning objective, loss
function, and initialization method, thereby can be regarded as a new paradigm
of single-step AT. Compared with other fast AT methods, TDAT can boost the
robustness of neural networks, alleviate the influence of misclassified
examples, and prevent CO during the training process while requiring almost no
additional computational and memory resources. Our method achieves robust
accuracy improvement of $1.59\%$, $1.62\%$, $0.71\%$, and $1.26\%$ on CIFAR-10,
CIFAR-100, Tiny ImageNet, and ImageNet-100 datasets, when against projected
gradient descent PGD10 attack with perturbation budget 8/255. Furthermore, our
proposed method also achieves state-of-the-art robust accuracy against other
attacks. Code is available at https://github.com/bookman233/TDAT.
