  We consider multivariate splines and show that they have a random feature
expansion as infinitely wide neural networks with one-hidden layer and a
homogeneous activation function which is the power of the rectified linear
unit. We show that the associated function space is a Sobolev space on a
Euclidean ball, with an explicit bound on the norms of derivatives. This link
provides a new random feature expansion for multivariate splines that allow
efficient algorithms. This random feature expansion is numerically better
behaved than usual random Fourier features, both in theory and practice. In
particular, in dimension one, we compare the associated leverage scores to
compare the two random expansions and show a better scaling for the neural
network expansion.
