  Overfitting in linear regression is broken down into two main causes. First,
the formula for the estimator includes 'forbidden knowledge' about training
observations' residuals, and it loses this advantage when deployed
out-of-sample. Second, the estimator has 'specialized training' that makes it
particularly capable of explaining movements in the predictors that are
idiosyncratic to the training sample. An out-of-sample counterpart is
introduced to the popular 'leverage' measure of training observations'
importance. A new method is proposed to forecast out-of-sample fit at the time
of deployment, when the values for the predictors are known but the true
outcome variable is not. In Monte Carlo simulations and in an empirical
application using MRI brain scans, the proposed estimator performs comparably
to Predicted Residual Error Sum of Squares (PRESS) for the average
out-of-sample case and unlike PRESS, also performs consistently across
different test samples, even those that differ substantially from the training
set.
