  Ensemble learning serves as a straightforward way to improve the performance
of almost any machine learning algorithm. Existing deep ensemble methods
usually naively train many different models and then aggregate their
predictions. This is not optimal in our view from two aspects: i) Naively
training multiple models adds much more computational burden, especially in the
deep learning era; ii) Purely optimizing each base model without considering
their interactions limits the diversity of ensemble and performance gains. We
tackle these issues by proposing deep negative correlation classification
(DNCC), in which the accuracy and diversity trade-off is systematically
controlled by decomposing the loss function seamlessly into individual accuracy
and the correlation between individual models and the ensemble. DNCC yields a
deep classification ensemble where the individual estimator is both accurate
and negatively correlated. Thanks to the optimized diversities, DNCC works well
even when utilizing a shared network backbone, which significantly improves its
efficiency when compared with most existing ensemble systems. Extensive
experiments on multiple benchmark datasets and network structures demonstrate
the superiority of the proposed method.
