  The relationship between the number of training data points, the number of
parameters, and the generalization capabilities has been widely studied.
Previous work has shown that double descent can occur in the over-parameterized
regime, and believe that the standard bias-variance trade-off holds in the
under-parameterized regime. These works provide multiple reasons for the
existence of the peak. We postulate that the location of the peak depends on
the technical properties of both the spectrum as well as the eigenvectors of
the sample covariance. We present two simple examples that provably exhibit
double descent in the under-parameterized regime and do not seem to occur for
reasons provided in prior work.
