  We introduce a discriminative regression approach to supervised
classification in this paper. It estimates a representation model while
accounting for discriminativeness between classes, thereby enabling accurate
derivation of categorical information. This new type of regression models
extends existing models such as ridge, lasso, and group lasso through
explicitly incorporating discriminative information. As a special case we focus
on a quadratic model that admits a closed-form analytical solution. The
corresponding classifier is called discriminative regression machine (DRM).
Three iterative algorithms are further established for the DRM to enhance the
efficiency and scalability for real applications. Our approach and the
algorithms are applicable to general types of data including images,
high-dimensional data, and imbalanced data. We compare the DRM with currently
state-of-the-art classifiers. Our extensive experimental results show superior
performance of the DRM and confirm the effectiveness of the proposed approach.
