  Big data is transforming our world, revolutionizing operations and analytics
everywhere, from financial engineering to biomedical sciences. The complexity
of big data often makes dimension reduction techniques necessary before
conducting statistical inference. Principal component analysis, commonly
referred to as PCA, has become an essential tool for multivariate data analysis
and unsupervised dimension reduction, the goal of which is to find a lower
dimensional subspace that captures most of the variation in the dataset. This
article provides an overview of methodological and theoretical developments of
PCA over the last decade, with focus on its applications to big data analytics.
We first review the mathematical formulation of PCA and its theoretical
development from the view point of perturbation analysis. We then briefly
discuss the relationship between PCA and factor analysis as well as its
applications to large covariance estimation and multiple testing. PCA also
finds important applications in many modern machine learning problems, and we
focus on community detection, ranking, mixture model and manifold learning in
this paper.
