  Dimension reduction algorithms aim to discover latent variables which
describe underlying structures in high-dimensional data. Methods such as factor
analysis and principal component analysis have the downside of not offering
much interpretability of its inferred latent variables. Sparse factor analysis
addresses this issue by imposing sparsity on its factor loadings, allowing each
latent variable to be related to only a subset of features, thus increasing
interpretability. Sparse factor analysis has been used in a wide range of areas
including genomics, signal processing, and economics. We compare two Bayesian
inference techniques for sparse factor analysis, namely Markov chain Monte
Carlo (MCMC), and variational inference (VI). VI is computationally faster than
MCMC, at the cost of a loss in accuracy. We derive MCMC and VI algorithms and
perform a comparison using both simulated and biological data, demonstrating
that the higher computational efficiency of VI is desirable over the small gain
in accuracy when using MCMC. Our implementation of MCMC and VI algorithms for
sparse factor analysis is available at https://github.com/ysfoo/sparsefactor.
