  This paper presents new algorithms to solve the feature-sparsity constrained
PCA problem (FSPCA), which performs feature selection and PCA simultaneously.
Existing optimization methods for FSPCA require data distribution assumptions
and are lack of global convergence guarantee. Though the general FSPCA problem
is NP-hard, we show that, for a low-rank covariance, FSPCA can be solved
globally (Algorithm 1). Then, we propose another strategy (Algorithm 2) to
solve FSPCA for the general covariance by iteratively building a carefully
designed proxy. We prove theoretical guarantees on approximation and
convergence for the new algorithms. Experimental results show the promising
performance of the new algorithms compared with the state-of-the-arts on both
synthetic and real-world datasets.
