  One way of getting insight into non-Gaussian measures, posed on infinite
dimensional Hilbert spaces, is to first obtain best fit Gaussian
approximations, which are more amenable to numerical approximation. These
Gaussians can then be used to accelerate sampling algorithms. This begs the
questions of how one should measure optimality and how the optimizers can be
obtained. Here, we consider the problem of minimizing the distance with respect
to relative entropy. We examine this minimization problem by seeking roots of
the first variation of relative entropy, taken with respect to the mean of the
Gaussian, leaving the covariance fixed. Adapting a convergence analysis of
Robbins-Monro to the infinite dimensional setting, we can justify the
application of this algorithm and highlight necessary assumptions to ensure
convergence, not only in the context of relative entropy minimization, but
other infinite dimensional problems as well. Numerical examples in path space,
showing the robustness of this method with respect to dimension, are provided.
