  We study the dynamics of an online algorithm for learning a sparse leading
eigenvector from samples generated from a spiked covariance model. This
algorithm combines the classical Oja's method for online PCA with an
element-wise nonlinearity at each iteration to promote sparsity. In the
high-dimensional limit, the joint empirical measure of the underlying sparse
eigenvector and its estimate provided by the algorithm is shown to converge
weakly to a deterministic, measure-valued process. This scaling limit is
characterized as the unique solution of a nonlinear PDE, and it provides exact
information regarding the asymptotic performance of the algorithm. For example,
performance metrics such as the cosine similarity and the misclassification
rate in sparse support recovery can be obtained by examining the limiting
dynamics. A steady-state analysis of the nonlinear PDE also reveals an
interesting phase transition phenomenon. Although our analysis is asymptotic in
nature, numerical simulations show that the theoretical predictions are
accurate for moderate signal dimensions.
