  Conditional Gaussian graphical models (cGGM) are a recent reparametrization
of the multivariate linear regression model which explicitly exhibits $i)$ the
partial covariances between the predictors and the responses, and $ii)$ the
partial covariances between the responses themselves. Such models are
particularly suitable for interpretability since partial covariances describe
strong relationships between variables. In this framework, we propose a
regularization scheme to enhance the learning strategy of the model by driving
the selection of the relevant input features by prior structural information.
It comes with an efficient alternating optimization procedure which is
guaranteed to converge to the global minimum. On top of showing competitive
performance on artificial and real datasets, our method demonstrates
capabilities for fine interpretation of its parameters, as illustrated on three
high-dimensional datasets from spectroscopy, genetics, and genomics.
