  Modern machine learning classifiers often exhibit vanishing classification
error on the training set. They achieve this by learning nonlinear
representations of the inputs that maps the data into linearly separable
classes.
  Motivated by these phenomena, we revisit high-dimensional maximum margin
classification for linearly separable data. We consider a stylized setting in
which data $(y_i,{\boldsymbol x}_i)$, $i\le n$ are i.i.d. with ${\boldsymbol
x}_i\sim\mathsf{N}({\boldsymbol 0},{\boldsymbol \Sigma})$ a $p$-dimensional
Gaussian feature vector, and $y_i \in\{+1,-1\}$ a label whose distribution
depends on a linear combination of the covariates $\langle {\boldsymbol
\theta}_*,{\boldsymbol x}_i \rangle$. While the Gaussian model might appear
extremely simplistic, universality arguments can be used to show that the
results derived in this setting also apply to the output of certain nonlinear
featurization maps.
  We consider the proportional asymptotics $n,p\to\infty$ with $p/n\to \psi$,
and derive exact expressions for the limiting generalization error. We use this
theory to derive two results of independent interest: $(i)$ Sufficient
conditions on $({\boldsymbol \Sigma},{\boldsymbol \theta}_*)$ for `benign
overfitting' that parallel previously derived conditions in the case of linear
regression; $(ii)$ An asymptotically exact expression for the generalization
error when max-margin classification is used in conjunction with feature
vectors produced by random one-layer neural networks.
