  High-dimensional data analysis has motivated a spectrum of regularization
methods for variable selection and sparse modeling, with two popular classes of
convex ones and concave ones. A long debate has been on whether one class
dominates the other, an important question both in theory and to practitioners.
In this paper, we characterize the asymptotic equivalence of regularization
methods, with general penalty functions, in a thresholded parameter space under
the generalized linear model setting, where the dimensionality can grow up to
exponentially with the sample size. To assess their performance, we establish
the oracle inequalities, as in Bickel, Ritov and Tsybakov (2009), of the global
minimizer for these methods under various prediction and variable selection
losses. These results reveal an interesting phase transition phenomenon. For
polynomially growing dimensionality, the $L_1$-regularization method of Lasso
and concave methods are asymptotically equivalent, having the same convergence
rates in the oracle inequalities. For exponentially growing dimensionality,
concave methods are asymptotically equivalent but have faster convergence rates
than the Lasso. We also establish a stronger property of the oracle risk
inequalities of the regularization methods, as well as the sampling properties
of computable solutions. Our new theoretical results are illustrated and
justified by simulation and real data examples.
