  Gaussian Processes (GP) are widely used for probabilistic modeling and
inference for nonparametric regression. However, their computational complexity
scales cubicly with the sample size rendering them unfeasible for large data
sets. To speed up the computations various distributed methods were proposed in
the literature. These methods have, however, limited theoretical underpinning.
In our work we derive frequentist theoretical guarantees and limitations for a
range of distributed methods for general GP priors in context of the
nonparametric regression model, both for recovery and uncertainty
quantification. As specific examples we consider covariance kernels both with
polynomially and exponentially decaying eigenvalues. We demonstrate the
practical performance of the investigated approaches in a numerical study using
synthetic data sets.
