  We propose a novel integrated formulation for multiclass and multilabel
support vector machines (SVMs). A number of approaches have been proposed to
extend the original binary SVM to an all-in-one multiclass SVM. However, its
direct extension to a unified multilabel SVM has not been widely investigated.
We propose a straightforward extension to the SVM to cope with multiclass and
multilabel classification problems within a unified framework. Our framework
deviates from the conventional soft margin SVM framework with its direct
oppositional structure. In our formulation, class-specific weight vectors
(normal vectors) are learned by maximizing their margin with respect to an
origin and penalizing patterns when they get too close to this origin. As a
result, each weight vector chooses an orientation and a magnitude with respect
to this origin in such a way that it best represents the patterns belonging to
its corresponding class. Opposition between classes is introduced into the
formulation via the minimization of pairwise inner products of weight vectors.
We also extend our framework to cope with nonlinear separability via standard
reproducing kernel Hilbert spaces (RKHS). Biases which are closely related to
the origin need to be treated properly in both the original feature space and
Hilbert space. We have the flexibility to incorporate constraints into the
formulation (if they better reflect the underlying geometry) and improve the
performance of the classifier. To this end, specifics and technicalities such
as the origin in RKHS are addressed. Results demonstrates a competitive
classifier for both multiclass and multilabel classification problems.
