  We propose a new Stein self-repulsive dynamics for obtaining diversified
samples from intractable un-normalized distributions. Our idea is to introduce
Stein variational gradient as a repulsive force to push the samples of Langevin
dynamics away from the past trajectories. This simple idea allows us to
significantly decrease the auto-correlation in Langevin dynamics and hence
increase the effective sample size. Importantly, as we establish in our
theoretical analysis, the asymptotic stationary distribution remains correct
even with the addition of the repulsive force, thanks to the special properties
of the Stein variational gradient. We perform extensive empirical studies of
our new algorithm, showing that our method yields much higher sample efficiency
and better uncertainty estimation than vanilla Langevin dynamics.
