  This paper advocates proximal Markov Chain Monte Carlo (ProxMCMC) as a
flexible and general Bayesian inference framework for constrained or
regularized estimation. Originally introduced in the Bayesian imaging
literature, ProxMCMC employs the Moreau-Yosida envelope for a smooth
approximation of the total-variation regularization term, fixes variance and
regularization strength parameters as constants, and uses the Langevin
algorithm for the posterior sampling. We extend ProxMCMC to be fully Bayesian
by providing data-adaptive estimation of all parameters including the
regularization strength parameter. More powerful sampling algorithms such as
Hamiltonian Monte Carlo are employed to scale ProxMCMC to high-dimensional
problems. Analogous to the proximal algorithms in optimization, ProxMCMC offers
a versatile and modularized procedure for conducting statistical inference on
constrained and regularized problems. The power of ProxMCMC is illustrated on
various statistical estimation and machine learning tasks, the inference of
which is traditionally considered difficult from both frequentist and Bayesian
perspectives.
