  In recent years, differential privacy has been adopted by tech-companies and
governmental agencies as the standard for measuring privacy in algorithms. We
study the level of differential privacy in Bayesian posterior sampling setups.
As opposed to the common privatization approach of injecting Laplace/Gaussian
noise into the output, Huber's contamination model is considered, where we
replace at random the data points with samples from a heavy-tailed
distribution. We derived bounds for the differential privacy level
$(\epsilon,\delta)$ for our approach while lifting the common restriction on
assuming bounded observation and parameter space seen in the existing
literature. We further consider the effect of sample size on privacy level and
the convergence rate of $(\epsilon,\delta)$ to zero. Asymptotically, the
contamination approach is fully private at no cost of information loss. We also
provide some examples depicting inference models that our setup is applicable
to with a theoretical estimation of convergence rate.
