  The methodology developed in this article is motivated by a wide range of
prediction and uncertainty quantification problems that arise in Statistics,
Machine Learning and Applied Mathematics, such as non-parametric regression,
multi-class classification and inversion of partial differential equations. One
popular formulation of such problems is as Bayesian inverse problems, where a
prior distribution is used to regularize inference on a high-dimensional latent
state, typically a function or a field. It is common that such priors are
non-Gaussian, for example piecewise-constant or heavy-tailed, and/or
hierarchical, in the sense of involving a further set of low-dimensional
parameters, which, for example, control the scale or smoothness of the latent
state. In this formulation prediction and uncertainty quantification relies on
efficient exploration of the posterior distribution of latent states and
parameters. This article introduces a framework for efficient MCMC sampling in
Bayesian inverse problems that capitalizes upon two fundamental ideas in MCMC,
non-centred parameterisations of hierarchical models and dimension-robust
samplers for latent Gaussian processes. Using a range of diverse applications
we showcase that the proposed framework is dimension-robust, that is, the
efficiency of the MCMC sampling does not deteriorate as the dimension of the
latent state gets higher. We showcase the full potential of the machinery we
develop in the article in semi-supervised multi-class classification, where our
sampling algorithm is used within an active learning framework to guide the
selection of input data to manually label in order to achieve high predictive
accuracy with a minimal number of labelled data.
