  The stochastic partial differential equation approach to Gaussian processes
(GPs) represents Mat\'ern GP priors in terms of $n$ finite element basis
functions and Gaussian coefficients with sparse precision matrix. Such
representations enhance the scalability of GP regression and classification to
datasets of large size $N$ by setting $n\approx N$ and exploiting sparsity. In
this paper we reconsider the standard choice $n \approx N$ through an analysis
of the estimation performance. Our theory implies that, under certain
smoothness assumptions, one can reduce the computation and memory cost without
hindering the estimation accuracy by setting $n \ll N$ in the large $N$
asymptotics. Numerical experiments illustrate the applicability of our theory
and the effect of the prior lengthscale in the pre-asymptotic regime.
