  We prove a universality theorem for learning with random features. Our result
shows that, in terms of training and generalization errors, a random feature
model with a nonlinear activation function is asymptotically equivalent to a
surrogate linear Gaussian model with a matching covariance matrix. This settles
a so-called Gaussian equivalence conjecture based on which several recent
papers develop their results. Our method for proving the universality theorem
builds on the classical Lindeberg approach. Major ingredients of the proof
include a leave-one-out analysis for the optimization problem associated with
the training process and a central limit theorem, obtained via Stein's method,
for weakly correlated random variables.
