  Deep Gaussian Processes (DGPs) leverage a compositional structure to model
non-stationary processes. DGPs typically rely on local inducing point
approximations across intermediate GP layers. Recent advances in DGP inference
have shown that incorporating global Fourier features from Reproducing Kernel
Hilbert Space (RKHS) can enhance the DGPs' capability to capture complex
non-stationary patterns. This paper extends the use of these features to
compositional GPs involving linear transformations. In particular, we introduce
Ordinary Differential Equation (ODE) -based RKHS Fourier features that allow
for adaptive amplitude and phase modulation through convolution operations.
This convolutional formulation relates our work to recently proposed deep
latent force models, a multi-layer structure designed for modelling nonlinear
dynamical systems. By embedding these adjustable RKHS Fourier features within a
doubly stochastic variational inference framework, our model exhibits improved
predictive performance across various regression tasks.
