  Sparse principal component analysis (SPCA) is widely used for dimensionality
reduction and feature extraction in high-dimensional data analysis. Despite
many methodological and theoretical developments in the past two decades, the
theoretical guarantees of the popular SPCA algorithm proposed by Zou, Hastie &
Tibshirani (2006) are still unknown. This paper aims to address this critical
gap. We first revisit the SPCA algorithm of Zou et al. (2006) and present our
implementation. We also study a computationally more efficient variant of the
SPCA algorithm in Zou et al. (2006) that can be considered as the limiting case
of SPCA. We provide the guarantees of convergence to a stationary point for
both algorithms and prove that, under a sparse spiked covariance model, both
algorithms can recover the principal subspace consistently under mild
regularity conditions. We show that their estimation error bounds match the
best available bounds of existing works or the minimax rates up to some
logarithmic factors. Moreover, we demonstrate the competitive numerical
performance of both algorithms in numerical studies.
