  We propose a unifying view of two different Bayesian inference algorithms,
Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) and Stein Variational
Gradient Descent (SVGD), leading to improved and efficient novel sampling
schemes. We show that SVGD combined with a noise term can be framed as a
multiple chain SG-MCMC method. Instead of treating each parallel chain
independently from others, our proposed algorithm implements a repulsive force
between particles, avoiding collapse and facilitating a better exploration of
the parameter space. We also show how the addition of this noise term is
necessary to obtain a valid SG-MCMC sampler, a significant difference with
SVGD. Experiments with both synthetic distributions and real datasets
illustrate the benefits of the proposed scheme.
