  Adversarial attack is commonly regarded as a huge threat to neural networks
because of misleading behavior. This paper presents an opposite perspective:
adversarial attacks can be harnessed to improve neural models if amended
correctly. Unlike traditional adversarial defense or adversarial training
schemes that aim to improve the adversarial robustness, the proposed
adversarial amendment (AdvAmd) method aims to improve the original accuracy
level of neural models on benign samples. We thoroughly analyze the
distribution mismatch between the benign and adversarial samples. This
distribution mismatch and the mutual learning mechanism with the same learning
ratio applied in prior art defense strategies is the main cause leading the
accuracy degradation for benign samples. The proposed AdvAmd is demonstrated to
steadily heal the accuracy degradation and even leads to a certain accuracy
boost of common neural models on benign classification, object detection, and
segmentation tasks. The efficacy of the AdvAmd is contributed by three key
components: mediate samples (to reduce the influence of distribution mismatch
with a fine-grained amendment), auxiliary batch norm (to solve the mutual
learning mechanism and the smoother judgment surface), and AdvAmd loss (to
adjust the learning ratios according to different attack vulnerabilities)
through quantitative and ablation experiments.
