  Stein variational gradient descent (SVGD) is a particle-based inference
algorithm that leverages gradient information for efficient approximate
inference. In this work, we enhance SVGD by leveraging preconditioning
matrices, such as the Hessian and Fisher information matrix, to incorporate
geometric information into SVGD updates. We achieve this by presenting a
generalization of SVGD that replaces the scalar-valued kernels in vanilla SVGD
with more general matrix-valued kernels. This yields a significant extension of
SVGD, and more importantly, allows us to flexibly incorporate various
preconditioning matrices to accelerate the exploration in the probability
landscape. Empirical results show that our method outperforms vanilla SVGD and
a variety of baseline approaches over a range of real-world Bayesian inference
tasks.
