  This paper studies the problem of estimating a large coefficient matrix in a
multiple response linear regression model when the coefficient matrix could be
both of low rank and sparse in the sense that most nonzero entries concentrate
on a few rows and columns. We are especially interested in the high dimensional
settings where the number of predictors and/or response variables can be much
larger than the number of observations. We propose a new estimation scheme,
which achieves competitive numerical performance and at the same time allows
fast computation. Moreover, we show that (a slight variant of) the proposed
estimator achieves near optimal non-asymptotic minimax rates of estimation
under a collection of squared Schatten norm losses simultaneously by providing
both the error bounds for the estimator and minimax lower bounds. The
effectiveness of the proposed algorithm is also demonstrated on an \textit{in
vivo} calcium imaging dataset.
