  We study the problem of supervised learning for both binary and multiclass
classification from a unified geometric perspective. In particular, we propose
a geometric regularization technique to find the submanifold corresponding to a
robust estimator of the class probability $P(y|\pmb{x})$. The regularization
term measures the volume of this submanifold, based on the intuition that
overfitting produces rapid local oscillations and hence large volume of the
estimator. This technique can be applied to regularize any classification
function that satisfies two requirements: firstly, an estimator of the class
probability can be obtained; secondly, first and second derivatives of the
class probability estimator can be calculated. In experiments, we apply our
regularization technique to standard loss functions for classification, our
RBF-based implementation compares favorably to widely used regularization
methods for both binary and multiclass classification.
