  The asymptotic optimality (a.o.) of various hyper-parameter estimators with
different optimality criteria has been studied in the literature for
regularized least squares regression problems. The estimators include e.g., the
maximum (marginal) likelihood method, $C_p$ statistics, and generalized cross
validation method, and the optimality criteria are based on e.g., the
inefficiency, the expectation inefficiency and the risk. In this paper, we
consider the regularized least squares regression problems with fixed number of
regression parameters, choose the optimality criterion based on the risk, and
study the a.o. of several cross validation (CV) based hyper-parameter
estimators including the leave $k$-out CV method, generalized CV method,
$r$-fold CV method and hold out CV method. We find the former three methods can
be a.o. under mild assumptions, but not the last one, and we use Monte Carlo
simulations to illustrate the efficacy of our findings.
