  Multivariate regression models are widely used in various fields such as
biology and finance. In this paper, we focus on two key challenges: (a) When
should we favor a multivariate model over a series of univariate models; (b) If
the numbers of responses and predictors are allowed to greatly exceed the
sample size, how to reduce the computational cost and provide precise
estimation. The proposed method, Interaction Pursuit Biconvex Optimization
(IPBO), explores the regression relationship allowing the predictors and
responses derived from different multivariate normal distributions with general
covariance matrices. In practice, the correlation structures within are complex
and interact on each other based on the regression function. The proposed
method solves this problem by building a structured sparsity penalty to
encourages the shared structure between the network and the regression
coefficients. We prove theoretical results under interpretable conditions, and
provide an efficient algorithm to compute the estimator. Simulation studies and
real data examples compare the proposed method with several existing methods,
indicating that IPBO works well.
