  The canonical algorithm for differentially private mean estimation is to
first clip the samples to a bounded range and then add noise to their empirical
mean. Clipping controls the sensitivity and, hence, the variance of the noise
that we add for privacy. But clipping also introduces statistical bias. We
prove that this tradeoff is inherent: no algorithm can simultaneously have low
bias, low variance, and low privacy loss for arbitrary distributions.
  On the positive side, we show that unbiased mean estimation is possible under
approximate differential privacy if we assume that the distribution is
symmetric. Furthermore, we show that, even if we assume that the data is
sampled from a Gaussian, unbiased mean estimation is impossible under pure or
concentrated differential privacy.
