  We propose a principal components regression method based on maximizing a
joint pseudo-likelihood for responses and predictors. Our method uses both
responses and predictors to select linear combinations of the predictors
relevant for the regression, thereby addressing an oft-cited deficiency of
conventional principal components regression. The proposed estimator is shown
to be consistent in a wide range of settings, including ones with non-normal
and dependent observations; conditions on the first and second moments suffice
if the number of predictors ($p$) is fixed and the number of observations ($n$)
tends to infinity and dependence is weak, while stronger distributional
assumptions are needed when $p \to \infty$ with $n$. We obtain the estimator's
asymptotic distribution as the projection of a multivariate normal random
vector onto a tangent cone of the parameter set at the true parameter, and find
the estimator is asymptotically more efficient than competing ones. In
simulations our method is substantially more accurate than conventional
principal components regression and compares favorably to partial least squares
and predictor envelopes. The method's practical usefulness is illustrated in a
data example with cross-sectional prediction of stock returns.
