  The Gaussian process state-space models (GPSSMs) represent a versatile class
of data-driven nonlinear dynamical system models. However, the presence of
numerous latent variables in GPSSM incurs unresolved issues for existing
variational inference approaches, particularly under the more realistic
non-mean-field (NMF) assumption, including extensive training effort,
compromised inference accuracy, and infeasibility for online applications,
among others. In this paper, we tackle these challenges by incorporating the
ensemble Kalman filter (EnKF), a well-established model-based filtering
technique, into the NMF variational inference framework to approximate the
posterior distribution of the latent states. This novel marriage between EnKF
and GPSSM not only eliminates the need for extensive parameterization in
learning variational distributions, but also enables an interpretable,
closed-form approximation of the evidence lower bound (ELBO). Moreover, owing
to the streamlined parameterization via the EnKF, the new GPSSM model can be
easily accommodated in online learning applications. We demonstrate that the
resulting EnKF-aided online algorithm embodies a principled objective function
by ensuring data-fitting accuracy while incorporating model regularizations to
mitigate overfitting. We also provide detailed analysis and fresh insights for
the proposed algorithms. Comprehensive evaluation across diverse real and
synthetic datasets corroborates the superior learning and inference performance
of our EnKF-aided variational inference algorithms compared to existing
methods.
