  Penalized (or regularized) regression, as represented by Lasso and its
variants, has become a standard technique for analyzing high-dimensional data
when the number of variables substantially exceeds the sample size. The
performance of penalized regression relies crucially on the choice of the
tuning parameter, which determines the amount of regularization and hence the
sparsity level of the fitted model. The optimal choice of tuning parameter
depends on both the structure of the design matrix and the unknown random error
distribution (variance, tail behavior, etc). This article reviews the current
literature of tuning parameter selection for high-dimensional regression from
both theoretical and practical perspectives. We discuss various strategies that
choose the tuning parameter to achieve prediction accuracy or support recovery.
We also review several recently proposed methods for tuning-free
high-dimensional regression.
