  Stochastic algorithms are efficient approaches to solving machine learning
and optimization problems. In this paper, we propose a general framework called
Splash for parallelizing stochastic algorithms on multi-node distributed
systems. Splash consists of a programming interface and an execution engine.
Using the programming interface, the user develops sequential stochastic
algorithms without concerning any detail about distributed computing. The
algorithm is then automatically parallelized by a communication-efficient
execution engine. We provide theoretical justifications on the optimal rate of
convergence for parallelizing stochastic gradient descent. Splash is built on
top of Apache Spark. The real-data experiments on logistic regression,
collaborative filtering and topic modeling verify that Splash yields
order-of-magnitude speedup over single-thread stochastic algorithms and over
state-of-the-art implementations on Spark.
