  A privacy-utility tradeoff is developed for an arbitrary set of
finite-alphabet source distributions. Privacy is quantified using differential
privacy (DP), and utility is quantified using expected Hamming distortion
maximized over the set of distributions. The family of source distribution sets
(source sets) is categorized into three classes, based on different levels of
prior knowledge they capture. For source sets whose convex hull includes the
uniform distribution, symmetric DP mechanisms are optimal. For source sets
whose probability values have a fixed monotonic ordering, asymmetric DP
mechanisms are optimal. For all other source sets, general upper and lower
bounds on the optimal privacy leakage are developed and a necessary and
sufficient condition for tightness are established. Differentially private
leakage is an upper bound on mutual information (MI) leakage: the two criteria
are compared analytically and numerically to illustrate the effect of adopting
a stronger privacy criterion.
