  In this paper, we consider multivariate response regression models with high
dimensional predictor variables. One way to model the correlation among the
response variables is through the low rank decomposition of the coefficient
matrix, which has been considered by several papers for the high dimensional
predictors. However, all these papers focus on the singular value decomposition
of the coefficient matrix. Our target is the decomposition of the coefficient
matrix which leads to the best lower rank approximation to the regression
function, the signal part in the response. Given any rank, this decomposition
has nearly the smallest expected prediction error among all approximations to
the the coefficient matrix with the same rank. To estimate the decomposition,
we formulate a penalized generalized eigenvalue problem to obtain the first
matrix in the decomposition and then obtain the second one by a least squares
method. In the high-dimensional setting, we establish the oracle inequalities
for the estimates. Compared to the existing theoretical results, we have less
restrictions on the distribution of the noise vector in each observation and
allow correlations among its coordinates. Our theoretical results do not depend
on the dimension of the multivariate response. Therefore, the dimension is
arbitrary and can be larger than the sample size and the dimension of the
predictor. Simulation studies and application to real data show that the
proposed method has good prediction performance and is efficient in dimension
reduction for various reduced rank models.
