  In this paper we study the problem of estimating the unknown mean $\theta$ of
a unit variance Gaussian distribution in a locally differentially private (LDP)
way. In the high-privacy regime ($\epsilon\le 0.67$), we identify the exact
optimal privacy mechanism that minimizes the variance of the estimator
asymptotically. It turns out to be the extraordinarily simple sign mechanism
that applies randomized response to the sign of $X_i-\theta$. However, since
this optimal mechanism depends on the unknown mean $\theta$, we employ a
two-stage LDP parameter estimation procedure which requires splitting agents
into two groups. The first $n_1$ observations are used to consistently but not
necessarily efficiently estimate the parameter $\theta$ by
$\tilde{\theta}_{n_1}$. Then this estimate is updated by applying the sign
mechanism with $\tilde{\theta}_{n_1}$ instead of $\theta$ to the remaining
$n-n_1$ observations, to obtain an LDP and efficient estimator of the unknown
mean.
