  This paper presents a differentially private algorithm for linear regression
learning in a decentralized fashion. Under this algorithm, privacy budget is
theoretically derived, in addition to that the solution error is shown to be
bounded by $O(t)$ for $O(1/t)$ descent step size and $O(\exp(t^{1-e}))$ for
$O(t^{-e})$ descent step size.
