  We propose a novel distributed inference algorithm for continuous graphical
models, by extending Stein variational gradient descent (SVGD) to leverage the
Markov dependency structure of the distribution of interest. Our approach
combines SVGD with a set of structured local kernel functions defined on the
Markov blanket of each node, which alleviates the curse of high dimensionality
and simultaneously yields a distributed algorithm for decentralized inference
tasks. We justify our method with theoretical analysis and show that the use of
local kernels can be viewed as a new type of localized approximation that
matches the target distribution on the conditional distributions of each node
over its Markov blanket. Our empirical results show that our method outperforms
a variety of baselines including standard MCMC and particle message passing
methods.
