  We propose a fast and scalable variational method for Bayesian inference in
high-dimensional parameter space, which we call projected Stein variational
Newton (pSVN) method. We exploit the intrinsic low-dimensional geometric
structure of the posterior distribution in the high-dimensional parameter space
via its Hessian (of the log posterior) operator and perform a parallel update
of the parameter samples projected into a low-dimensional subspace by an SVN
method. The subspace is adaptively constructed using the eigenvectors of the
averaged Hessian at the current samples. We demonstrate fast convergence of the
proposed method and its scalability with respect to the number of parameters,
samples, and processor cores.
