  Deep neural network ensembles combine the wisdom of multiple deep neural
networks to improve the generalizability and robustness over individual
networks. It has gained increasing popularity to study deep ensemble techniques
in the deep learning community. Some mission-critical applications utilize a
large number of deep neural networks to form deep ensembles to achieve desired
accuracy and resilience, which introduces high time and space costs for
ensemble execution. However, it still remains a critical challenge whether a
small subset of the entire deep ensemble can achieve the same or better
generalizability and how to effectively identify these small deep ensembles for
improving the space and time efficiency of ensemble execution. This paper
presents a novel deep ensemble pruning approach, which can efficiently identify
smaller deep ensembles and provide higher ensemble accuracy than the entire
deep ensemble of a large number of member networks. Our hierarchical ensemble
pruning approach (HQ) leverages three novel ensemble pruning techniques. First,
we show that the focal diversity metrics can accurately capture the
complementary capacity of the member networks of an ensemble, which can guide
ensemble pruning. Second, we design a focal diversity based hierarchical
pruning approach, which will iteratively find high quality deep ensembles with
low cost and high accuracy. Third, we develop a focal diversity consensus
method to integrate multiple focal diversity metrics to refine ensemble pruning
results, where smaller deep ensembles can be effectively identified to offer
high accuracy, high robustness and high efficiency. Evaluated using popular
benchmark datasets, we demonstrate that the proposed hierarchical ensemble
pruning approach can effectively identify high quality deep ensembles with
better generalizability while being more time and space efficient in ensemble
decision making.
