  I review some of the main methods for selecting tuning parameters in
nonparametric and $\ell_1$-penalized estimation. For the nonparametric
estimation, I consider the methods of Mallows, Stein, Lepski, cross-validation,
penalization, and aggregation in the context of series estimation. For the
$\ell_1$-penalized estimation, I consider the methods based on the theory of
self-normalized moderate deviations, bootstrap, Stein's unbiased risk
estimation, and cross-validation in the context of Lasso estimation. I explain
the intuition behind each of the methods and discuss their comparative
advantages. I also give some extensions.
