  Variational Bayesian inference is an important machine-learning tool that
finds application from statistics to robotics. The goal is to find an
approximate probability density function (PDF) from a chosen family that is in
some sense 'closest' to the full Bayesian posterior. Closeness is typically
defined through the selection of an appropriate loss functional such as the
Kullback-Leibler (KL) divergence. In this paper, we explore a new formulation
of variational inference by exploiting the fact that (most) PDFs are members of
a Bayesian Hilbert space under careful definitions of vector addition, scalar
multiplication and an inner product. We show that, under the right conditions,
variational inference based on KL divergence can amount to iterative
projection, in the Euclidean sense, of the Bayesian posterior onto a subspace
corresponding to the selected approximation family. We work through the details
of this general framework for the specific case of the Gaussian approximation
family and show the equivalence to another Gaussian variational inference
approach. We furthermore discuss the implications for systems that exhibit
sparsity, which is handled naturally in Bayesian space, and give an example of
a high-dimensional robotic state estimation problem that can be handled as a
result. We provide some preliminary examples of how the approach could be
applied to non-Gaussian inference and discuss the limitations of the approach
in detail to encourage follow-on work along these lines.
