  Deep Neural Networks are vulnerable to adversarial attacks even in settings
where the attacker has no direct access to the model being attacked. Such
attacks usually rely on the principle of transferability, whereby an attack
crafted on a surrogate model tends to transfer to the target model. We show
that an ensemble of models with misaligned loss gradients can provide an
effective defense against transfer-based attacks. Our key insight is that an
adversarial example is less likely to fool multiple models in the ensemble if
their loss functions do not increase in a correlated fashion. To this end, we
propose Diversity Training, a novel method to train an ensemble of models with
uncorrelated loss functions. We show that our method significantly improves the
adversarial robustness of ensembles and can also be combined with existing
methods to create a stronger defense.
