  Adversarial robustness has emerged as an important topic in deep learning as
carefully crafted attack samples can significantly disturb the performance of a
model. Many recent methods have proposed to improve adversarial robustness by
utilizing adversarial training or model distillation, which adds additional
procedures to model training. In this paper, we propose a new training paradigm
called Guided Complement Entropy (GCE) that is capable of achieving
"adversarial defense for free," which involves no additional procedures in the
process of improving adversarial robustness. In addition to maximizing model
probabilities on the ground-truth class like cross-entropy, we neutralize its
probabilities on the incorrect classes along with a "guided" term to balance
between these two terms. We show in the experiments that our method achieves
better model robustness with even better performance compared to the commonly
used cross-entropy training objective. We also show that our method can be used
orthogonal to adversarial training across well-known methods with noticeable
robustness gain. To the best of our knowledge, our approach is the first one
that improves model robustness without compromising performance.
