  We view the penalty algorithm of Ceperley and Dewing (1999), a Markov chain
Monte Carlo (MCMC) algorithm for Bayesian inference, in the context of data
privacy. Specifically, we study differential privacy of the penalty algorithm
and advocate its use for data privacy. We show that in the simple model of
independent observations the algorithm has desirable convergence and privacy
properties that scale with data size. Two special cases are also investigated
and privacy preserving schemes are proposed for those cases: (i) Data are
distributed among several data owners who are interested in the inference of a
common parameter while preserving their data privacy. (ii) The data likelihood
belongs to an exponential family.
