  Bayesian Additive Regression Trees (BART) is a Bayesian approach to flexible
non-linear regression which has been shown to be competitive with the best
modern predictive methods such as those based on bagging and boosting. BART
offers some advantages. For example, the stochastic search Markov Chain Monte
Carlo (MCMC) algorithm can provide a more complete search of the model space
and variation across MCMC draws can capture the level of uncertainty in the
usual Bayesian way. The BART prior is robust in that reasonable results are
typically obtained with a default prior specification. However, the publicly
available implementation of the BART algorithm in the R package BayesTree is
not fast enough to be considered interactive with over a thousand observations,
and is unlikely to even run with 50,000 to 100,000 observations. In this paper
we show how the BART algorithm may be modified and then computed using single
program, multiple data (SPMD) parallel computation implemented using the
Message Passing Interface (MPI) library. The approach scales nearly linearly in
the number of processor cores, enabling the practitioner to perform statistical
inference on massive datasets. Our approach can also handle datasets too
massive to fit on any single data repository.
