  This work considers black-box Bayesian inference over high-dimensional
parameter spaces. The well-known adaptive Metropolis (AM) algorithm of (Haario
etal. 2001) is extended herein to scale asymptotically uniformly with respect
to the underlying parameter dimension for Gaussian targets, by respecting the
variance of the target. The resulting algorithm, referred to as the
dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved
performance with respect to adaptive Metropolis on non-Gaussian targets. This
algorithm is further improved, and the possibility of probing high-dimensional
targets is enabled, via GPU-accelerated numerical libraries and periodically
synchronized concurrent chains (justified a posteriori). Asymptotically in
dimension, this GPU implementation exhibits a factor of four improvement versus
a competitive CPU-based Intel MKL parallel version alone. Strong scaling to
concurrent chains is exhibited, through a combination of longer time per sample
batch (weak scaling) and yet fewer necessary samples to convergence. The
algorithm performance is illustrated on several Gaussian and non-Gaussian
target examples, in which the dimension may be in excess of one thousand.
