  When the data are stored in a distributed manner, direct application of
traditional statistical inference procedures is often prohibitive due to
communication cost and privacy concerns. This paper develops and investigates
two Communication-Efficient Accurate Statistical Estimators (CEASE),
implemented through iterative algorithms for distributed optimization. In each
iteration, node machines carry out computation in parallel and communicate with
the central processor, which then broadcasts aggregated information to node
machines for new updates. The algorithms adapt to the similarity among loss
functions on node machines, and converge rapidly when each node machine has
large enough sample size. Moreover, they do not require good initialization and
enjoy linear converge guarantees under general conditions. The contraction rate
of optimization errors is presented explicitly, with dependence on the local
sample size unveiled. In addition, the improved statistical accuracy per
iteration is derived. By regarding the proposed method as a multi-step
statistical estimator, we show that statistical efficiency can be achieved in
finite steps in typical statistical applications. In addition, we give the
conditions under which the one-step CEASE estimator is statistically efficient.
Extensive numerical experiments on both synthetic and real data validate the
theoretical results and demonstrate the superior performance of our algorithms.
