  Partial least squares (PLS) regression combines dimensionality reduction and
prediction using a latent variable model. Since partial least squares
regression (PLS-R) does not require matrix inversion or diagonalization, it can
be applied to problems with large numbers of variables. As predictor dimension
increases, variable selection becomes essential to avoid over-fitting, to
provide more accurate predictors and to yield more interpretable parameters. We
propose a global variable selection approach that penalizes the total number of
variables across all PLS components. Put another way, the proposed global
penalty encourages the selected variables to be shared among the PLS
components. We formulate PLS-R with joint sparsity as a variational
optimization problem with objective function equal to a novel global SIMPLS
criterion plus a mixed norm sparsity penalty on the weight matrix. The mixed
norm sparsity penalty is the $\ell_1$ norm of the $\ell_2$ norm on the weights
corresponding to the same variable used over all the PLS components. A novel
augmented Lagrangian method is proposed to solve the optimization problem and
soft thresholding for sparsity occurs naturally as part of the iterative
solution. Experiments show that the modified PLS-R attains better or as good
performance with many fewer selected predictor variables.
