  Portfolio optimization requires sophisticated covariance estimators that are
able to filter out estimation noise. Non-linear shrinkage is a popular
estimator based on how the Oracle eigenvalues can be computed using only data
from the calibration window. Contrary to common belief, NLS is not optimal for
portfolio optimization because it does not minimize the right cost function
when the asset dependence structure is non-stationary. We instead derive the
optimal target. Using historical data, we quantify by how much NLS can be
improved. Our findings reopen the question of how to build the covariance
matrix estimator for portfolio optimization in realistic conditions.
