  The debiased estimator is a crucial tool in statistical inference for
high-dimensional model parameters. However, constructing such an estimator
involves estimating the high-dimensional inverse Hessian matrix, incurring
significant computational costs. This challenge becomes particularly acute in
distributed setups, where traditional methods necessitate computing a debiased
estimator on every machine. This becomes unwieldy, especially with a large
number of machines. In this paper, we delve into semi-supervised sparse
statistical inference in a distributed setup. An efficient multi-round
distributed debiased estimator, which integrates both labeled and unlabelled
data, is developed. We will show that the additional unlabeled data helps to
improve the statistical rate of each round of iteration. Our approach offers
tailored debiasing methods for $M$-estimation and generalized linear models
according to the specific form of the loss function. Our method also applies to
a non-smooth loss like absolute deviation loss. Furthermore, our algorithm is
computationally efficient since it requires only one estimation of a
high-dimensional inverse covariance matrix. We demonstrate the effectiveness of
our method by presenting simulation studies and real data applications that
highlight the benefits of incorporating unlabeled data.
