  In this paper, we study frequentist coverage errors of Bayesian credible sets
for an approximately linear regression model with (moderately) high dimensional
regressors, where the dimension of the regressors may increase with but is
smaller than the sample size. Specifically, we consider quasi-Bayesian
inference on the slope vector under the quasi-likelihood with Gaussian error
distribution. Under this setup, we derive finite sample bounds on frequentist
coverage errors of Bayesian credible rectangles. Derivation of those bounds
builds on a novel Berry--Esseen type bound on quasi-posterior distributions and
recent results on high-dimensional CLT on hyperrectangles. We use this general
result to quantify coverage errors of Castillo--Nickl and $L^{\infty}$-credible
bands for Gaussian white noise models, linear inverse problems, and (possibly
non-Gaussian) nonparametric regression models. In particular, we show that
Bayesian credible bands for those nonparametric models have coverage errors
decaying polynomially fast in the sample size, implying advantages of Bayesian
credible bands over confidence bands based on extreme value theory.
