  Machine learning algorithms have been effectively applied into various real
world tasks. However, it is difficult to provide high-quality machine learning
solutions to accommodate an unknown distribution of input datasets; this
difficulty is called the uncertainty prediction problems. In this paper, a
margin-based Pareto deep ensemble pruning (MBPEP) model is proposed. It
achieves the high-quality uncertainty estimation with a small value of the
prediction interval width (MPIW) and a high confidence of prediction interval
coverage probability (PICP) by using deep ensemble networks. In addition to
these networks, unique loss functions are proposed, and these functions make
the sub-learners available for standard gradient descent learning. Furthermore,
the margin criterion fine-tuning-based Pareto pruning method is introduced to
optimize the ensembles. Several experiments including predicting uncertainties
of classification and regression are conducted to analyze the performance of
MBPEP. The experimental results show that MBPEP achieves a small interval width
and a low learning error with an optimal number of ensembles. For the
real-world problems, MBPEP performs well on input datasets with unknown
distributions datasets incomings and improves learning performance on a multi
task problem when compared to that of each single model.
