  Ensemble methods such as bagging and random forests are ubiquitous in various
fields, from finance to genomics. Despite their prevalence, the question of the
efficient tuning of ensemble parameters has received relatively little
attention. This paper introduces a cross-validation method, ECV (Extrapolated
Cross-Validation), for tuning the ensemble and subsample sizes in randomized
ensembles. Our method builds on two primary ingredients: initial estimators for
small ensemble sizes using out-of-bag errors and a novel risk extrapolation
technique that leverages the structure of prediction risk decomposition. By
establishing uniform consistency of our risk extrapolation technique over
ensemble and subsample sizes, we show that ECV yields $\delta$-optimal (with
respect to the oracle-tuned risk) ensembles for squared prediction risk. Our
theory accommodates general ensemble predictors, only requires mild moment
assumptions, and allows for high-dimensional regimes where the feature
dimension grows with the sample size. As a practical case study, we employ ECV
to predict surface protein abundances from gene expressions in single-cell
multiomics using random forests. In comparison to sample-split cross-validation
and $K$-fold cross-validation, ECV achieves higher accuracy avoiding sample
splitting. At the same time, its computational cost is considerably lower owing
to the use of the risk extrapolation technique. Additional numerical results
validate the finite-sample accuracy of ECV for several common ensemble
predictors under a computational constraint on the maximum ensemble size.
