  We examine an analytic variational inference scheme for the Gaussian Process
State Space Model (GPSSM) - a probabilistic model for system identification and
time-series modelling. Our approach performs variational inference over both
the system states and the transition function. We exploit Markov structure in
the true posterior, as well as an inducing point approximation to achieve
linear time complexity in the length of the time series. Contrary to previous
approaches, no Monte Carlo sampling is required: inference is cast as a
deterministic optimisation problem. In a number of experiments, we demonstrate
the ability to model non-linear dynamics in the presence of both process and
observation noise as well as to impute missing information (e.g. velocities
from raw positions through time), to de-noise, and to estimate the underlying
dimensionality of the system. Finally, we also introduce a closed-form method
for multi-step prediction, and a novel criterion for assessing the quality of
our approximate posterior.
