  We study general coordinate-wise MCMC schemes (such as
Metropolis-within-Gibbs samplers), which are commonly used to fit Bayesian
non-conjugate hierarchical models. We relate their convergence properties to
the ones of the corresponding (potentially not implementable) Gibbs sampler
through the notion of conditional conductance. This allows us to study the
performances of popular Metropolis-within-Gibbs schemes for non-conjugate
hierarchical models, in high-dimensional regimes where both number of
datapoints and parameters increase. Given random data-generating assumptions,
we establish dimension-free convergence results, which are in close accordance
with numerical evidences. Applications to Bayesian models for binary regression
with unknown hyperparameters and discretely observed diffusions are also
discussed. Motivated by such statistical applications, auxiliary results of
independent interest on approximate conductances and perturbation of Markov
operators are provided.
