  In high dimensional regression, where the number of covariates is of the
order of the number of observations, ridge penalization is often used as a
remedy against overfitting. Unfortunately, for correlated covariates such
regularisation typically induces in generalized linear models not only
shrinking of the estimated parameter vector, but also an unwanted
\emph{rotation} relative to the true vector. We show analytically how this
problem can be removed by using a generalization of ridge penalization, and we
analyse the asymptotic properties of the corresponding estimators in the high
dimensional regime, using the cavity method. Our results also provide a
quantitative rationale for tuning the parameter that controlling the amount of
shrinking. We compare our theoretical predictions with simulated data and find
excellent agreement.
