  Gaussian processes are a key component of many flexible statistical and
machine learning models. However, they exhibit cubic computational complexity
and high memory constraints due to the need of inverting and storing a full
covariance matrix. To circumvent this, mixtures of Gaussian process experts
have been considered where data points are assigned to independent experts,
reducing the complexity by allowing inference based on smaller, local
covariance matrices. Moreover, mixtures of Gaussian process experts
substantially enrich the model's flexibility, allowing for behaviors such as
non-stationarity, heteroscedasticity, and discontinuities. In this work, we
construct a novel inference approach based on nested sequential Monte Carlo
samplers to simultaneously infer both the gating network and Gaussian process
expert parameters. This greatly improves inference compared to importance
sampling, particularly in settings when a stationary Gaussian process is
inappropriate, while still being thoroughly parallelizable.
