  The integration of an ensemble of deep learning models has been extensively
explored to enhance defense against adversarial attacks. The diversity among
sub-models increases the attack cost required to deceive the majority of the
ensemble, thereby improving the adversarial robustness. While existing
approaches mainly center on increasing diversity in feature representations or
dispersion of first-order gradients with respect to input, the limited
correlation between these diversity metrics and adversarial robustness
constrains the performance of ensemble adversarial defense. In this work, we
aim to enhance ensemble diversity by reducing attack transferability. We
identify second-order gradients, which depict the loss curvature, as a key
factor in adversarial robustness. Computing the Hessian matrix involved in
second-order gradients is computationally expensive. To address this, we
approximate the Hessian-vector product using differential approximation. Given
that low curvature provides better robustness, our ensemble model was designed
to consider the influence of curvature among different sub-models. We introduce
a novel regularizer to train multiple more-diverse low-curvature network
models. Extensive experiments across various datasets demonstrate that our
ensemble model exhibits superior robustness against a range of attacks,
underscoring the effectiveness of our approach.
