  Adversarial Propagation (AdvProp) is an effective way to improve recognition
models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the
extremely slow training speed, mainly because: a) extra forward and backward
passes are required for generating adversarial examples; b) both original
samples and their adversarial counterparts are used for training (i.e.,
2$\times$ data). In this paper, we introduce Fast AdvProp, which aggressively
revamps AdvProp's costly training components, rendering the method nearly as
cheap as the vanilla training. Specifically, our modifications in Fast AdvProp
are guided by the hypothesis that disentangled learning with adversarial
examples is the key for performance improvements, while other training recipes
(e.g., paired clean and adversarial training samples, multi-step adversarial
attackers) could be largely simplified.
  Our empirical results show that, compared to the vanilla training baseline,
Fast AdvProp is able to further model performance on a spectrum of visual
benchmarks, without incurring extra training cost. Additionally, our ablations
find Fast AdvProp scales better if larger models are used, is compatible with
existing data augmentation methods (i.e., Mixup and CutMix), and can be easily
adapted to other recognition tasks like object detection. The code is available
here: https://github.com/meijieru/fast_advprop.
