  We study the problem of learning an unknown function using random feature
models. Our main contribution is an exact asymptotic analysis of such learning
problems with Gaussian data. Under mild regularity conditions for the feature
matrix, we provide an exact characterization of the asymptotic training and
generalization errors, valid in both the under-parameterized and
over-parameterized regimes. The analysis presented in this paper holds for
general families of feature matrices, activation functions, and convex loss
functions. Numerical results validate our theoretical predictions, showing that
our asymptotic findings are in excellent agreement with the actual performance
of the considered learning problem, even in moderate dimensions. Moreover, they
reveal an important role played by the regularization, the loss function and
the activation function in the mitigation of the "double descent phenomenon" in
learning.
