  While great progress has been made at making neural networks effective across
a wide range of visual tasks, most models are surprisingly vulnerable. This
frailness takes the form of small, carefully chosen perturbations of their
input, known as adversarial examples, which represent a security threat for
learned vision models in the wild -- a threat which should be responsibly
defended against in safety-critical applications of computer vision. In this
paper, we advocate for and experimentally investigate the use of a family of
logit regularization techniques as an adversarial defense, which can be used in
conjunction with other methods for creating adversarial robustness at little to
no marginal cost. We also demonstrate that much of the effectiveness of one
recent adversarial defense mechanism can in fact be attributed to logit
regularization, and show how to improve its defense against both white-box and
black-box attacks, in the process creating a stronger black-box attack against
PGD-based models. We validate our methods on three datasets and include results
on both gradient-free attacks and strong gradient-based iterative attacks with
as many as 1,000 steps.
