  The modern scale of data has brought new challenges to Bayesian inference. In
particular, conventional MCMC algorithms are computationally very expensive for
large data sets. A promising approach to solve this problem is embarrassingly
parallel MCMC (EP-MCMC), which first partitions the data into multiple subsets
and runs independent sampling algorithms on each subset. The subset posterior
draws are then aggregated via some combining rules to obtain the final
approximation. Existing EP-MCMC algorithms are limited by approximation
accuracy and difficulty in resampling. In this article, we propose a new
EP-MCMC algorithm PART that solves these problems. The new algorithm applies
random partition trees to combine the subset posterior draws, which is
distribution-free, easy to resample from and can adapt to multiple scales. We
provide theoretical justification and extensive experiments illustrating
empirical performance.
