  Sparse variational Gaussian process (SVGP) methods are a common choice for
non-conjugate Gaussian process inference because of their computational
benefits. In this paper, we improve their computational efficiency by using a
dual parameterization where each data example is assigned dual parameters,
similarly to site parameters used in expectation propagation. Our dual
parameterization speeds-up inference using natural gradient descent, and
provides a tighter evidence lower bound for hyperparameter learning. The
approach has the same memory cost as the current SVGP methods, but it is faster
and more accurate.
