  We study a setting where a data holder wishes to share data with a receiver,
without revealing certain summary statistics of the data distribution (e.g.,
mean, standard deviation). It achieves this by passing the data through a
randomization mechanism. We propose summary statistic privacy, a metric for
quantifying the privacy risk of such a mechanism based on the worst-case
probability of an adversary guessing the distributional secret within some
threshold. Defining distortion as a worst-case Wasserstein-1 distance between
the real and released data, we prove lower bounds on the tradeoff between
privacy and distortion. We then propose a class of quantization mechanisms that
can be adapted to different data distributions. We show that the quantization
mechanism's privacy-distortion tradeoff matches our lower bounds under certain
regimes, up to small constant factors. Finally, we demonstrate on real-world
datasets that the proposed quantization mechanisms achieve better
privacy-distortion tradeoffs than alternative privacy mechanisms.
