  The rapid emergence of massive datasets in various fields poses a serious
challenge to traditional statistical methods. Meanwhile, it provides
opportunities for researchers to develop novel algorithms. Inspired by the idea
of divide-and-conquer, various distributed frameworks for statistical
estimation and inference have been proposed. They were developed to deal with
large-scale statistical optimization problems. This paper aims to provide a
comprehensive review for related literature. It includes parametric models,
nonparametric models, and other frequently used models. Their key ideas and
theoretical properties are summarized. The trade-off between communication cost
and estimate precision together with other concerns are discussed.
