  Sparse Bayesian factor models are routinely implemented for parsimonious
dependence modeling and dimensionality reduction in high-dimensional
applications. We provide theoretical understanding of such Bayesian procedures
in terms of posterior convergence rates in inferring high-dimensional
covariance matrices where the dimension can be larger than the sample size.
Under relevant sparsity assumptions on the true covariance matrix, we show that
commonly-used point mass mixture priors on the factor loadings lead to
consistent estimation in the operator norm even when $p\gg n$. One of our major
contributions is to develop a new class of continuous shrinkage priors and
provide insights into their concentration around sparse vectors. Using such
priors for the factor loadings, we obtain similar rate of convergence as
obtained with point mass mixture priors. To obtain the convergence rates, we
construct test functions to separate points in the space of high-dimensional
covariance matrices using insights from random matrix theory; the tools
developed may be of independent interest. We also derive minimax rates and show
that the Bayesian posterior rates of convergence coincide with the minimax
rates upto a $\sqrt{\log n}$ term.
