  In this paper, we study efficient differentially private alternating
direction methods of multipliers (ADMM) via gradient perturbation for many
machine learning problems. For smooth convex loss functions with (non)-smooth
regularization, we propose the first differentially private ADMM (DP-ADMM)
algorithm with performance guarantee of $(\epsilon,\delta)$-differential
privacy ($(\epsilon,\delta)$-DP). From the viewpoint of theoretical analysis,
we use the Gaussian mechanism and the conversion relationship between R\'enyi
Differential Privacy (RDP) and DP to perform a comprehensive privacy analysis
for our algorithm. Then we establish a new criterion to prove the convergence
of the proposed algorithms including DP-ADMM. We also give the utility analysis
of our DP-ADMM. Moreover, we propose an accelerated DP-ADMM (DP-AccADMM) with
the Nesterov's acceleration technique. Finally, we conduct numerical
experiments on many real-world datasets to show the privacy-utility tradeoff of
the two proposed algorithms, and all the comparative analysis shows that
DP-AccADMM converges faster and has a better utility than DP-ADMM, when the
privacy budget $\epsilon$ is larger than a threshold.
