  It has been recognized that the diversity of base learners is of utmost
importance to a good ensemble. This paper defines a novel measurement of
diversity, termed as exclusivity. With the designed exclusivity, we further
propose an ensemble model, namely Exclusivity Regularized Machine (ERM), to
jointly suppress the training error of ensemble and enhance the diversity
between bases. Moreover, an Augmented Lagrange Multiplier based algorithm is
customized to effectively and efficiently seek the optimal solution of ERM.
Theoretical analysis on convergence and global optimality of the proposed
algorithm, as well as experiments are provided to reveal the efficacy of our
method and show its superiority over state-of-the-art alternatives in terms of
accuracy and efficiency.
