  Ensemble models refer to methods that combine a typically large number of
classifiers into a compound prediction. The output of an ensemble method is the
result of fitting a base-learning algorithm to a given data set, and obtaining
diverse answers by reweighting the observations or by resampling them using a
given probabilistic selection. A key challenge of using ensembles in
large-scale multidimensional data lies in the complexity and the computational
burden associated with them. The models created by ensembles are often
difficult, if not impossible, to interpret and their implementation requires
more computational power than single classifiers. Recent research effort in the
field has concentrated in reducing ensemble size, while maintaining their
predictive accuracy. We propose a method to prune an ensemble solution by
optimizing its margin distribution, while increasing its diversity. The
proposed algorithm results in an ensemble that uses only a fraction of the
original classifiers, with improved or similar generalization performance. We
analyze and test our method on both synthetic and real data sets. The
simulations show that the proposed method compares favorably to the original
ensemble solutions and to other existing ensemble pruning methodologies.
