  Bayesian factor analysis is routinely used for dimensionality reduction in
modeling of high-dimensional covariance matrices. Factor analytic
decompositions express the covariance as a sum of a low rank and diagonal
matrix. In practice, Gibbs sampling algorithms are typically used for posterior
computation, alternating between updating the latent factors, loadings, and
residual variances. In this article, we exploit a blessing of dimensionality to
develop a provably accurate pseudo-posterior for the covariance matrix that
bypasses the need for Gibbs or other variants of Markov chain Monte Carlo
sampling. Our proposed Factor Analysis with BLEssing of dimensionality (FABLE)
approach relies on a first-stage singular value decomposition (SVD) to estimate
the latent factors, and then defines a jointly conjugate prior for the loadings
and residual variances. The accuracy of the resulting pseudo-posterior for the
covariance improves with increasing dimensionality. We show that FABLE has
excellent performance in high-dimensional covariance matrix estimation,
including producing well calibrated credible intervals, both theoretically and
through simulation experiments. We also demonstrate the strength of our
approach in terms of accurate inference and computational efficiency by
applying it to a gene expression data set.
