  Efficiently accessing the information contained in non-linear and high
dimensional probability distributions remains a core challenge in modern
statistics. Traditionally, estimators that go beyond point estimates are either
categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC)
techniques. While MCMC methods that utilize the geometric properties of
continuous probability distributions to increase their efficiency have been
proposed, VI methods rarely use the geometry. This work aims to fill this gap
and proposes geometric Variational Inference (geoVI), a method based on
Riemannian geometry and the Fisher information metric. It is used to construct
a coordinate transformation that relates the Riemannian manifold associated
with the metric to Euclidean space. The distribution, expressed in the
coordinate system induced by the transformation, takes a particularly simple
form that allows for an accurate variational approximation by a normal
distribution. Furthermore, the algorithmic structure allows for an efficient
implementation of geoVI which is demonstrated on multiple examples, ranging
from low-dimensional illustrative ones to non-linear, hierarchical Bayesian
inverse problems in thousands of dimensions.
