  The performance of penalized likelihood approaches depends profoundly on the
selection of the tuning parameter; however, there is no commonly agreed-upon
criterion for choosing the tuning parameter. Moreover, penalized likelihood
estimation based on a single value of the tuning parameter suffers from several
drawbacks. This article introduces a novel approach for feature selection based
on the entire solution paths rather than the choice of a single tuning
parameter, which significantly improves the accuracy of the selection.
Moreover, the approach allows for feature selection using ridge or other
strictly convex penalties. The key idea is to classify variables as relevant or
irrelevant at each tuning parameter and then to select all of the variables
which have been classified as relevant at least once. We establish the
theoretical properties of the method, which requires significantly weaker
conditions than existing methods in the literature. We also illustrate the
advantages of the proposed approach with simulation studies and a data example.
