  The growth in the use of computationally intensive statistical procedures,
especially with Big Data, has necessitated the usage of parallel computation on
diverse platforms such as multicore, GPU, clusters and clouds. However,
slowdown due to interprocess communication costs typically limits such methods
to "embarrassingly parallel" (EP) algorithms, especially on non-shared memory
platforms. This paper develops a broadly-applicable method for converting many
non-EP algorithms into statistically equivalent EP ones. The method is shown to
yield excellent levels of speedup for a variety of statistical computations. It
also overcomes certain problems of memory limitations.
