  Support vector machine is an important and fundamental technique in machine
learning. Soft-margin SVM models have stronger generalization performance
compared with the hard-margin SVM. Most existing works use the hinge-loss
function which can be regarded as an upper bound of the 0-1 loss function.
However, it can not explicitly limit the number of misclassified samples. In
this paper, we use the idea of soft-margin SVM and propose a new SVM model with
a sparse constraint. Our model can strictly limit the number of misclassified
samples, expressing the soft-margin constraint as a sparse constraint. By
constructing a majorization function, a majorization penalty method can be used
to solve the sparse-constrained optimization problem. We apply
Conjugate-Gradient (CG) method to solve the resulting subproblem. Extensive
numerical results demonstrate the impressive performance of the proposed
majorization penalty method.
