  We formulate approximate Bayesian inference in non-conjugate temporal and
spatio-temporal Gaussian process models as a simple parameter update rule
applied during Kalman smoothing. This viewpoint encompasses most inference
schemes, including expectation propagation (EP), the classical (Extended,
Unscented, etc.) Kalman smoothers, and variational inference. We provide a
unifying perspective on these algorithms, showing how replacing the power EP
moment matching step with linearisation recovers the classical smoothers. EP
provides some benefits over the traditional methods via introduction of the
so-called cavity distribution, and we combine these benefits with the
computational efficiency of linearisation, providing extensive empirical
analysis demonstrating the efficacy of various algorithms under this unifying
framework. We provide a fast implementation of all methods in JAX.
