  We investigate the HSIC (Hilbert-Schmidt independence criterion) bottleneck
as a regularizer for learning an adversarially robust deep neural network
classifier. In addition to the usual cross-entropy loss, we add regularization
terms for every intermediate layer to ensure that the latent representations
retain useful information for output prediction while reducing redundant
information. We show that the HSIC bottleneck enhances robustness to
adversarial attacks both theoretically and experimentally. In particular, we
prove that the HSIC bottleneck regularizer reduces the sensitivity of the
classifier to adversarial examples. Our experiments on multiple benchmark
datasets and architectures demonstrate that incorporating an HSIC bottleneck
regularizer attains competitive natural accuracy and improves adversarial
robustness, both with and without adversarial examples during training. Our
code and adversarially robust models are publicly available.
