  We propose a general method for distributed Bayesian model choice, using the
marginal likelihood, where a data set is split in non-overlapping subsets.
These subsets are only accessed locally by individual workers and no data is
shared between the workers. We approximate the model evidence for the full data
set through Monte Carlo sampling from the posterior on every subset generating
a model evidence per subset. The results are combined using a novel approach
which corrects for the splitting using summary statistics of the generated
samples. Our divide-and-conquer approach enables Bayesian model choice in the
large data setting, exploiting all available information but limiting
communication between workers. We derive theoretical error bounds that quantify
the resulting trade-off between computational gain and loss in precision. The
embarrassingly parallel nature yields important speed-ups when used on massive
data sets as illustrated by our real world experiments. In addition, we show
how the suggested approach can be extended to model choice within a reversible
jump setting that explores multiple feature combinations within one run.
