  Due to the dominant position of deep learning (mostly deep neural networks)
in various artificial intelligence applications, recently, ensemble learning
based on deep neural networks (ensemble deep learning) has shown significant
performances in improving the generalization of learning system. However, since
modern deep neural networks usually have millions to billions of parameters,
the time and space overheads for training multiple base deep learners and
testing with the ensemble deep learner are far greater than that of traditional
ensemble learning. Though several algorithms of fast ensemble deep learning
have been proposed to promote the deployment of ensemble deep learning in some
applications, further advances still need to be made for many applications in
specific fields, where the developing time and computing resources are usually
restricted or the data to be processed is of large dimensionality. An urgent
problem needs to be solved is how to take the significant advantages of
ensemble deep learning while reduce the required expenses so that many more
applications in specific fields can benefit from it. For the alleviation of
this problem, it is essential to know about how ensemble learning has developed
under the era of deep learning. Thus, in this article, we present fundamental
discussions focusing on data analyses of published works, methodologies, recent
advances and unattainability of traditional ensemble learning and ensemble deep
learning. We hope this article will be helpful to realize the intrinsic
problems and technical challenges faced by future developments of ensemble
learning under the era of deep learning.
