  Since its early use in least squares regression problems, the l1-penalization
framework for variable selection has been employed in conjunction with a wide
range of loss functions encompassing regression, classification and survival
analysis. While a well developed theory exists for the l1-penalized least
squares estimates, few results concern the behavior of l1-penalized estimates
for general loss functions. In this paper, we derive two results concerning
penalized estimates for a wide array of penalty and loss functions. Our first
result characterizes the asymptotic distribution of penalized parametric
M-estimators under mild conditions on the loss and penalty functions in the
classical setting (fixed-p-large-n). Our second result explicits necessary and
sufficient generalized irrepresentability (GI) conditions for l1-penalized
parametric M-estimates to consistently select the components of a model
(sparsistency) as well as their sign (sign consistency). In general, the GI
conditions depend on the Hessian of the risk function at the true value of the
unknown parameter. Under Gaussian predictors, we obtain a set of conditions
under which the GI conditions can be re-expressed solely in terms of the second
moment of the predictors. We apply our theory to contrast l1-penalized SVM and
logistic regression classifiers and find conditions under which they have the
same behavior in terms of their model selection consistency (sparsistency and
sign consistency). Finally, we provide simulation evidence for the theory based
on these classification examples.
