  Iteratively reweighted least squares (IRLS) is a widely-used method in
machine learning to estimate the parameters in the generalised linear models.
In particular, IRLS for L1 minimisation under the linear model provides a
closed-form solution in each step, which is a simple multiplication between the
inverse of the weighted second moment matrix and the weighted first moment
vector. When dealing with privacy sensitive data, however, developing a privacy
preserving IRLS algorithm faces two challenges. First, due to the inversion of
the second moment matrix, the usual sensitivity analysis in differential
privacy incorporating a single datapoint perturbation gets complicated and
often requires unrealistic assumptions. Second, due to its iterative nature, a
significant cumulative privacy loss occurs. However, adding a high level of
noise to compensate for the privacy loss hinders from getting accurate
estimates. Here, we develop a practical algorithm that overcomes these
challenges and outputs privatised and accurate IRLS solutions. In our method,
we analyse the sensitivity of each moments separately and treat the matrix
inversion and multiplication as a post-processing step, which simplifies the
sensitivity analysis. Furthermore, we apply the {\it{concentrated differential
privacy}} formalism, a more relaxed version of differential privacy, which
requires adding a significantly less amount of noise for the same level of
privacy guarantee, compared to the conventional and advanced compositions of
differentially private mechanisms.
