  Large-scale association analysis between multivariate responses and
predictors is of great practical importance, as exemplified by modern business
applications including social media marketing and crisis management. Despite
the rapid methodological advances, how to obtain scalable estimators with free
tuning of the regularization parameters remains unclear under general noise
covariance structures. In this paper, we develop a new methodology called
sequential scaled sparse factor regression (SESS) based on a new viewpoint that
the problem of recovering a jointly low-rank and sparse regression coefficient
matrix can be decomposed into several univariate response sparse regressions
through regular eigenvalue decomposition. It combines the strengths of
sequential estimation and scaled sparse regression, thus sharing the
scalability and the tuning free property for sparsity parameters inherited from
the two approaches. The stepwise convex formulation, sequential factor
regression framework, and tuning insensitiveness make SESS highly scalable for
big data applications. Comprehensive theoretical justifications with new
insights into high-dimensional multi-response regressions are also provided. We
demonstrate the scalability and effectiveness of the proposed method by
simulation studies and stock short interest data analysis.
