  Non-Euclidean spaces routinely arise in modern statistical applications such
as in medical imaging, robotics, and computer vision, to name a few. While
traditional Bayesian approaches are applicable to such settings by considering
an ambient Euclidean space as the parameter space, we demonstrate the benefits
of integrating manifold structure into the Bayesian framework, both
theoretically and computationally. Moreover, existing Bayesian approaches which
are designed specifically for manifold-valued parameters are primarily
model-based, which are typically subject to inaccurate uncertainty
quantification under model misspecification. In this article, we propose a
robust model-free Bayesian inference for parameters defined on a Riemannian
submanifold, which is shown to provide valid uncertainty quantification from a
frequentist perspective. Computationally, we propose a Markov chain Monte Carlo
to sample from the posterior on the Riemannian submanifold, where the mixing
time, in the large sample regime, is shown to depend only on the intrinsic
dimension of the parameter space instead of the potentially much larger ambient
dimension. Our numerical results demonstrate the effectiveness of our approach
on a variety of problems, such as reduced-rank multiple quantile regression,
principal component analysis, and Fr\'{e}chet mean estimation.
