  (Gradient) Expectation Maximization (EM) is a widely used algorithm for
estimating the maximum likelihood of mixture models or incomplete data
problems. A major challenge facing this popular technique is how to effectively
preserve the privacy of sensitive data. Previous research on this problem has
already lead to the discovery of some Differentially Private (DP) algorithms
for (Gradient) EM. However, unlike in the non-private case, existing techniques
are not yet able to provide finite sample statistical guarantees. To address
this issue, we propose in this paper the first DP version of (Gradient) EM
algorithm with statistical guarantees. Moreover, we apply our general framework
to three canonical models: Gaussian Mixture Model (GMM), Mixture of Regressions
Model (MRM) and Linear Regression with Missing Covariates (RMC). Specifically,
for GMM in the DP model, our estimation error is near optimal in some cases.
For the other two models, we provide the first finite sample statistical
guarantees. Our theory is supported by thorough numerical experiments.
