  The study of stability and sensitivity of statistical methods or algorithms
with respect to their data is an important problem in machine learning and
statistics. The performance of the algorithm under resampling of the data is a
fundamental way to measure its stability and is closely related to
generalization or privacy of the algorithm. In this paper, we study the
resampling sensitivity for the principal component analysis (PCA). Given an $ n
\times p $ random matrix $ \mathbf{X} $, let $ \mathbf{X}^{[k]} $ be the matrix
obtained from $ \mathbf{X} $ by resampling $ k $ randomly chosen entries of $
\mathbf{X} $. Let $ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ denote the principal
components of $ \mathbf{X} $ and $ \mathbf{X}^{[k]} $. In the proportional
growth regime $ p/n \to \xi \in (0,1] $, we establish the sharp threshold for
the sensitivity/stability transition of PCA. When $ k \gg n^{5/3} $, the
principal components $ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ are asymptotically
orthogonal. On the other hand, when $ k \ll n^{5/3} $, the principal components
$ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ are asymptotically colinear. In words,
we show that PCA is sensitive to the input data in the sense that resampling
even a negligible portion of the input may completely change the output.
