  Support vector machine (SVM) has been one of the most popular learning
algorithms, with the central idea of maximizing the minimum margin, i.e., the
smallest distance from the instances to the classification boundary. Recent
theoretical results, however, disclosed that maximizing the minimum margin does
not necessarily lead to better generalization performances, and instead, the
margin distribution has been proven to be more crucial. Based on this idea, we
propose a new method, named Optimal margin Distribution Machine (ODM), which
tries to achieve a better generalization performance by optimizing the margin
distribution. We characterize the margin distribution by the first- and
second-order statistics, i.e., the margin mean and variance. The proposed
method is a general learning approach which can be used in any place where SVM
can be applied, and their superiority is verified both theoretically and
empirically in this paper.
