  Our goal is to understand why the robustness drops after conducting
adversarial training for too long. Although this phenomenon is commonly
explained as overfitting, our analysis suggest that its primary cause is
perturbation underfitting. We observe that after training for too long,
FGSM-generated perturbations deteriorate into random noise. Intuitively, since
no parameter updates are made to strengthen the perturbation generator, once
this process collapses, it could be trapped in such local optima. Also,
sophisticating this process could mostly avoid the robustness drop, which
supports that this phenomenon is caused by underfitting instead of overfitting.
In the light of our analyses, we propose APART, an adaptive adversarial
training framework, which parameterizes perturbation generation and
progressively strengthens them. Shielding perturbations from underfitting
unleashes the potential of our framework. In our experiments, APART provides
comparable or even better robustness than PGD-10, with only about 1/4 of its
computational cost.
