  Modern applications of Bayesian inference involve models that are
sufficiently complex that the corresponding posterior distributions are
intractable and must be approximated. The most common approximation is based on
Markov chain Monte Carlo, but these can be expensive when the data set is large
and/or the model is complex, so more efficient variational approximations have
recently received considerable attention. The traditional variational methods,
that seek to minimize the Kullback--Leibler divergence between the posterior
and a relatively simple parametric family, provide accurate and efficient
estimation of the posterior mean, but often does not capture other moments, and
have limitations in terms of the models to which they can be applied. Here we
propose the construction of variational approximations based on minimizing the
Fisher divergence, and develop an efficient computational algorithm that can be
applied to a wide range of models without conjugacy or potentially unrealistic
mean-field assumptions. We demonstrate the superior performance of the proposed
method for the benchmark case of logistic regression.
