  We study the distributions of the LASSO, SCAD, and thresholding estimators,
in finite samples and in the large-sample limit. The asymptotic distributions
are derived for both the case where the estimators are tuned to perform
consistent model selection and for the case where the estimators are tuned to
perform conservative model selection. Our findings complement those of Knight
and Fu (2000) and Fan and Li (2001). We show that the distributions are
typically highly nonnormal regardless of how the estimator is tuned, and that
this property persists in large samples. The uniform convergence rate of these
estimators is also obtained, and is shown to be slower than 1/root(n) in case
the estimator is tuned to perform consistent model selection. An impossibility
result regarding estimation of the estimators' distribution function is also
provided.
