  In this paper, we study adversarial training on datasets that obey the
long-tailed distribution, which is practical but rarely explored in previous
works. Compared with conventional adversarial training on balanced datasets,
this process falls into the dilemma of generating uneven adversarial examples
(AEs) and an unbalanced feature embedding space, causing the resulting model to
exhibit low robustness and accuracy on tail data. To combat that, we
theoretically analyze the lower bound of the robust risk to train a model on a
long-tailed dataset to obtain the key challenges in addressing the
aforementioned dilemmas. Based on it, we propose a new adversarial training
framework -- Re-balancing Adversarial Training (REAT). This framework consists
of two components: (1) a new training strategy inspired by the effective number
to guide the model to generate more balanced and informative AEs; (2) a
carefully constructed penalty function to force a satisfactory feature space.
Evaluation results on different datasets and model structures prove that REAT
can effectively enhance the model's robustness and preserve the model's clean
accuracy. The code can be found in https://github.com/GuanlinLee/REAT.
