  In the context of adversarial robustness, a single model does not usually
have enough power to defend against all possible adversarial attacks, and as a
result, has sub-optimal robustness. Consequently, an emerging line of work has
focused on learning an ensemble of neural networks to defend against
adversarial attacks. In this work, we take a principled approach towards
building robust ensembles. We view this problem from the perspective of
margin-boosting and develop an algorithm for learning an ensemble with maximum
margin. Through extensive empirical evaluation on benchmark datasets, we show
that our algorithm not only outperforms existing ensembling techniques, but
also large models trained in an end-to-end fashion. An important byproduct of
our work is a margin-maximizing cross-entropy (MCE) loss, which is a better
alternative to the standard cross-entropy (CE) loss. Empirically, we show that
replacing the CE loss in state-of-the-art adversarial training techniques with
our MCE loss leads to significant performance improvement.
