  Gaussian processes are ubiquitous in machine learning, statistics, and
applied mathematics. They provide a flexible modelling framework for
approximating functions, whilst simultaneously quantifying uncertainty.
However, this is only true when the model is well-specified, which is often not
the case in practice. In this paper, we study the properties of Gaussian
process means when the smoothness of the model and the likelihood function are
misspecified. In this setting, an important theoretical question of practial
relevance is how accurate the Gaussian process approximations will be given the
difficulty of the problem, our model and the extent of the misspecification.
The answer to this problem is particularly useful since it can inform our
choice of model and experimental design. In particular, we describe how the
experimental design and choice of kernel and kernel hyperparameters can be
adapted to alleviate model misspecification.
