  We derive upper bounds on the Wasserstein distance ($W_1$), with respect to
$\sup$-norm, between any continuous $\mathbb{R}^d$ valued random field indexed
by the $n$-sphere and the Gaussian, based on Stein's method. We develop a novel
Gaussian smoothing technique that allows us to transfer a bound in a smoother
metric to the $W_1$ distance. The smoothing is based on covariance functions
constructed using powers of Laplacian operators, designed so that the
associated Gaussian process has a tractable Cameron-Martin or Reproducing
Kernel Hilbert Space. This feature enables us to move beyond one dimensional
interval-based index sets that were previously considered in the literature.
Specializing our general result, we obtain the first bounds on the Gaussian
random field approximation of wide random neural networks of any depth and
Lipschitz activation functions at the random field level. Our bounds are
explicitly expressed in terms of the widths of the network and moments of the
random weights. We also obtain tighter bounds when the activation function has
three bounded derivatives.
