  We tackle the problem of estimating a location parameter with differential
privacy guarantees and sub-Gaussian deviations. Recent work in statistics has
focused on the study of estimators that achieve sub-Gaussian type deviations
even for heavy tailed data. We revisit some of these estimators through the
lens of differential privacy and show that a naive application of the Laplace
mechanism can lead to sub-optimal results. We design two private algorithms for
estimating the median that lead to estimators with sub-Gaussian type errors.
Unlike most existing differentially private median estimators, both algorithms
are well defined for unbounded random variables that are not even required to
have finite moments. We then turn to the problem of sub-Gaussian mean
estimation and show that under heavy tails natural differentially private
alternatives lead to strictly worse deviations than their non-private
sub-Gaussian counterparts. This is in sharp contrast with recent results that
show that from an asymptotic perspective the cost of differential privacy is
negligible.
