  A range of defense methods have been proposed to improve the robustness of
neural networks on adversarial examples, among which provable defense methods
have been demonstrated to be effective to train neural networks that are
certifiably robust to the attacker. However, most of these provable defense
methods treat all examples equally during training process, which ignore the
inconsistent constraint of certified robustness between correctly classified
(natural) and misclassified examples. In this paper, we explore this
inconsistency caused by misclassified examples and add a novel consistency
regularization term to make better use of the misclassified examples.
Specifically, we identified that the certified robustness of network can be
significantly improved if the constraint of certified robustness on
misclassified examples and correctly classified examples is consistent.
Motivated by this discovery, we design a new defense regularization term called
Misclassification Aware Adversarial Regularization (MAAR), which constrains the
output probability distributions of all examples in the certified region of the
misclassified example. Experimental results show that our proposed MAAR
achieves the best certified robustness and comparable accuracy on CIFAR-10 and
MNIST datasets in comparison with several state-of-the-art methods.
