  Deep neural networks are vulnerable to adversarial examples. Adversarial
training (AT) is an effective defense against adversarial examples. However, AT
is prone to overfitting which degrades robustness substantially. Recently, data
augmentation (DA) was shown to be effective in mitigating robust overfitting if
appropriately designed and optimized for AT. This work proposes a new method to
automatically learn online, instance-wise, DA policies to improve robust
generalization for AT. A novel policy learning objective, consisting of
Vulnerability, Affinity and Diversity, is proposed and shown to be sufficiently
effective and efficient to be practical for automatic DA generation during AT.
This allows our method to efficiently explore a large search space for a more
effective DA policy and evolve the policy as training progresses. Empirically,
our method is shown to outperform or match all competitive DA methods across
various model architectures (CNNs and ViTs) and datasets (CIFAR10, SVHN and
Imagenette). Our DA policy reinforced vanilla AT to surpass several
state-of-the-art AT methods (with baseline DA) in terms of both accuracy and
robustness. It can also be combined with those advanced AT methods to produce a
further boost in robustness.
