  This thesis is mainly concerned with state-space approaches for solving deep
(temporal) Gaussian process (DGP) regression problems. More specifically, we
represent DGPs as hierarchically composed systems of stochastic differential
equations (SDEs), and we consequently solve the DGP regression problem by using
state-space filtering and smoothing methods. The resulting state-space DGP
(SS-DGP) models generate a rich class of priors compatible with modelling a
number of irregular signals/functions. Moreover, due to their Markovian
structure, SS-DGPs regression problems can be solved efficiently by using
Bayesian filtering and smoothing methods. The second contribution of this
thesis is that we solve continuous-discrete Gaussian filtering and smoothing
problems by using the Taylor moment expansion (TME) method. This induces a
class of filters and smoothers that can be asymptotically exact in predicting
the mean and covariance of stochastic differential equations (SDEs) solutions.
Moreover, the TME method and TME filters and smoothers are compatible with
simulating SS-DGPs and solving their regression problems. Lastly, this thesis
features a number of applications of state-space (deep) GPs. These applications
mainly include, (i) estimation of unknown drift functions of SDEs from
partially observed trajectories and (ii) estimation of spectro-temporal
features of signals.
