  Sparse principal component analysis (SPCA) is a popular tool for
dimensionality reduction in high-dimensional data. However, there is still a
lack of theoretically justified Bayesian SPCA methods that can scale well
computationally. One of the major challenges in Bayesian SPCA is selecting an
appropriate prior for the loadings matrix, considering that principal
components are mutually orthogonal. We propose a novel parameter-expanded
coordinate ascent variational inference (PX-CAVI) algorithm. This algorithm
utilizes a spike and slab prior, which incorporates parameter expansion to cope
with the orthogonality constraint. Besides comparing to two popular SPCA
approaches, we introduce the PX-EM algorithm as an EM analogue to the PX-CAVI
algorithm for comparison. Through extensive numerical simulations, we
demonstrate that the PX-CAVI algorithm outperforms these SPCA approaches,
showcasing its superiority in terms of performance. We study the posterior
contraction rate of the variational posterior, providing a novel contribution
to the existing literature. The PX-CAVI algorithm is then applied to study a
lung cancer gene expression dataset. The R package VBsparsePCA with an
implementation of the algorithm is available on the Comprehensive R Archive
Network (CRAN).
