  Cross-Validation (CV) is the default choice for evaluating the performance of
machine learning models. Despite its wide usage, their statistical benefits
have remained half-understood, especially in challenging nonparametric regimes.
In this paper we fill in this gap and show that in fact, for a wide spectrum of
models, CV does not statistically outperform the simple "plug-in" approach
where one reuses training data for testing evaluation. Specifically, in terms
of both the asymptotic bias and coverage accuracy of the associated interval
for out-of-sample evaluation, $K$-fold CV provably cannot outperform plug-in
regardless of the rate at which the parametric or nonparametric models
converge. Leave-one-out CV can have a smaller bias as compared to plug-in;
however, this bias improvement is negligible compared to the variability of the
evaluation, and in some important cases leave-one-out again does not outperform
plug-in once this variability is taken into account. We obtain our theoretical
comparisons via a novel higher-order Taylor analysis that allows us to derive
necessary conditions for limit theorems of testing evaluations, which applies
to model classes that are not amenable to previously known sufficient
conditions. Our numerical results demonstrate that plug-in performs indeed no
worse than CV across a wide range of examples.
