  Adversarial Training (AT) has become arguably the state-of-the-art algorithm
for extracting robust features. However, researchers recently notice that AT
suffers from severe robust overfitting problems, particularly after learning
rate (LR) decay. In this paper, we explain this phenomenon by viewing
adversarial training as a dynamic minimax game between the model trainer and
the attacker. Specifically, we analyze how LR decay breaks the balance between
the minimax game by empowering the trainer with a stronger memorization
ability, and show such imbalance induces robust overfitting as a result of
memorizing non-robust features. We validate this understanding with extensive
experiments, and provide a holistic view of robust overfitting from the
dynamics of both the two game players. This understanding further inspires us
to alleviate robust overfitting by rebalancing the two players by either
regularizing the trainer's capacity or improving the attack strength.
Experiments show that the proposed ReBalanced Adversarial Training (ReBAT) can
attain good robustness and does not suffer from robust overfitting even after
very long training. Code is available at https://github.com/PKU-ML/ReBAT.
