  In recent years, there is a growing need to train machine learning models on
a huge volume of data. Designing efficient distributed optimization algorithms
for empirical risk minimization (ERM) has therefore become an active and
challenging research topic. In this paper, we propose a flexible framework for
distributed ERM training through solving the dual problem, which provides a
unified description and comparison of existing methods. Our approach requires
only approximate solutions of the sub-problems involved in the optimization
process, and is versatile to be applied on many large-scale machine learning
problems including classification, regression, and structured prediction. We
show that our approach enjoys global linear convergence for a broader class of
problems, and achieves faster empirical performance, compared with existing
works.
