  Using methods of Statistical Physics, we investigate the generalization
performance of support vector machines (SVMs), which have been recently
introduced as a general alternative to neural networks. For nonlinear
classification rules, the generalization error saturates on a plateau, when the
number of examples is too small to properly estimate the coefficients of the
nonlinear part. When trained on simple rules, we find that SVMs overfit only
weakly. The performance of SVMs is strongly enhanced, when the distribution of
the inputs has a gap in feature space.
