  Sparse reduced-rank regression is an important tool to uncover meaningful
dependence structure between large numbers of predictors and responses in many
big data applications such as genome-wide association studies and social media
analysis. Despite the recent theoretical and algorithmic advances, scalable
estimation of sparse reduced-rank regression remains largely unexplored. In
this paper, we suggest a scalable procedure called sequential estimation with
eigen-decomposition (SEED) which needs only a single top-$r$ singular value
decomposition to find the optimal low-rank and sparse matrix by solving a
sparse generalized eigenvalue problem. Our suggested method is not only
scalable but also performs simultaneous dimensionality reduction and variable
selection. Under some mild regularity conditions, we show that SEED enjoys nice
sampling properties including consistency in estimation, rank selection,
prediction, and model selection. Numerical studies on synthetic and real data
sets show that SEED outperforms the state-of-the-art approaches for large-scale
matrix estimation problem.
