  Standard Markov chain Monte Carlo methods struggle to explore distributions
that are concentrated in the neighbourhood of low-dimensional structures. These
pathologies naturally occur in a number of situations. For example, they are
common to Bayesian inverse problem modelling and Bayesian neural networks, when
observational data are highly informative, or when a subset of the statistical
parameters of interest are non-identifiable. In this paper, we propose a
strategy that transforms the original sampling problem into the task of
exploring a distribution supported on a manifold embedded in a higher
dimensional space; in contrast to the original posterior this lifted
distribution remains diffuse in the vanishing noise limit. We employ a
constrained Hamiltonian Monte Carlo method which exploits the manifold geometry
of this lifted distribution, to perform efficient approximate inference. We
demonstrate in several numerical experiments that, contrarily to competing
approaches, the sampling efficiency of our proposed methodology does not
degenerate as the target distribution to be explored concentrates near low
dimensional structures.
