  Gaussian process (GP) priors are non-parametric generative models with
appealing modelling properties for Bayesian inference: they can model
non-linear relationships through noisy observations, have closed-form
expressions for training and inference, and are governed by interpretable
hyperparameters. However, GP models rely on Gaussianity, an assumption that
does not hold in several real-world scenarios, e.g., when observations are
bounded or have extreme-value dependencies, a natural phenomenon in physics,
finance and social sciences. Although beyond-Gaussian stochastic processes have
caught the attention of the GP community, a principled definition and rigorous
treatment is still lacking. In this regard, we propose a methodology to
construct stochastic processes, which include GPs, warped GPs, Student-t
processes and several others under a single unified approach. We also provide
formulas and algorithms for training and inference of the proposed models in
the regression problem. Our approach is inspired by layers-based models, where
each proposed layer changes a specific property over the generated stochastic
process. That, in turn, allows us to push-forward a standard Gaussian white
noise prior towards other more expressive stochastic processes, for which
marginals and copulas need not be Gaussian, while retaining the appealing
properties of GPs. We validate the proposed model through experiments with
real-world data.
