  Novel and high-performance medical image classification pipelines are heavily
utilizing ensemble learning strategies. The idea of ensemble learning is to
assemble diverse models or multiple predictions and, thus, boost prediction
performance. However, it is still an open question to what extent as well as
which ensemble learning strategies are beneficial in deep learning based
medical image classification pipelines. In this work, we proposed a
reproducible medical image classification pipeline for analyzing the
performance impact of the following ensemble learning techniques: Augmenting,
Stacking, and Bagging. The pipeline consists of state-of-the-art preprocessing
and image augmentation methods as well as 9 deep convolution neural network
architectures. It was applied on four popular medical imaging datasets with
varying complexity. Furthermore, 12 pooling functions for combining multiple
predictions were analyzed, ranging from simple statistical functions like
unweighted averaging up to more complex learning-based functions like support
vector machines. Our results revealed that Stacking achieved the largest
performance gain of up to 13% F1-score increase. Augmenting showed consistent
improvement capabilities by up to 4% and is also applicable to single model
based pipelines. Cross-validation based Bagging demonstrated significant
performance gain close to Stacking, which resulted in an F1-score increase up
to +11%. Furthermore, we demonstrated that simple statistical pooling functions
are equal or often even better than more complex pooling functions. We
concluded that the integration of ensemble learning techniques is a powerful
method for any medical image classification pipeline to improve robustness and
boost performance.
