  By reinterpreting a robust discriminative classifier as Energy-based Model
(EBM), we offer a new take on the dynamics of adversarial training (AT). Our
analysis of the energy landscape during AT reveals that untargeted attacks
generate adversarial images much more in-distribution (lower energy) than the
original data from the point of view of the model. Conversely, we observe the
opposite for targeted attacks. On the ground of our thorough analysis, we
present new theoretical and practical results that show how interpreting AT
energy dynamics unlocks a better understanding: (1) AT dynamic is governed by
three phases and robust overfitting occurs in the third phase with a drastic
divergence between natural and adversarial energies (2) by rewriting the loss
of TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization
(TRADES) in terms of energies, we show that TRADES implicitly alleviates
overfitting by means of aligning the natural energy with the adversarial one
(3) we empirically show that all recent state-of-the-art robust classifiers are
smoothing the energy landscape and we reconcile a variety of studies about
understanding AT and weighting the loss function under the umbrella of EBMs.
Motivated by rigorous evidence, we propose Weighted Energy Adversarial Training
(WEAT), a novel sample weighting scheme that yields robust accuracy matching
the state-of-the-art on multiple benchmarks such as CIFAR-10 and SVHN and going
beyond in CIFAR-100 and Tiny-ImageNet. We further show that robust classifiers
vary in the intensity and quality of their generative capabilities, and offer a
simple method to push this capability, reaching a remarkable Inception Score
(IS) and FID using a robust classifier without training for generative
modeling. The code to reproduce our results is available at
http://github.com/OmnAI-Lab/Robust-Classifiers-under-the-lens-of-EBM/ .
