  Distributed sparse learning for high dimensional parameters has attached vast
attentions due to its wide application in prediction and classification in
diverse fields of machine learning. Existing distributed sparse regression
usually takes an average way to ensemble the local results produced by
distributed machines, which enjoys low communication cost but is statistical
inefficient. To address this problem, we proposed a new Weighted AVerage
Estimate (WAVE) for high-dimensional regressions. The WAVE is a solution to a
weighted least-square loss with an adaptive $L_1$ penalty, in which the $L_1$
penalty controls the sparsity and the weight promotes the statistical
efficiency. It can not only achieve a balance between the statistical and
communication efficiency, but also reach a faster rate than the average
estimate with a very low communication cost, requiring the local machines
delivering two vectors to the master merely. The consistency of parameter
estimation and model selection is also provided, which guarantees the safety of
using WAVE in the distributed system. The consistency also provides a way to
make hypothisis testing on the parameter. Moreover, WAVE is robust to the
heterogeneous distributed samples with varied mean and covariance across
machines, which has been verified by the asymptotic normality under such
conditions. Other competitors, however, do not own this property. The
effectiveness of WAVE is further illustrated by extensive numerical studies and
real data analyses.
