  Recently, considerable interest has focused on variable selection methods in
regression situations where the number of predictors, $p$, is large relative to
the number of observations, $n$. Two commonly applied variable selection
approaches are the Lasso, which computes highly shrunk regression coefficients,
and Forward Selection, which uses no shrinkage. We propose a new approach,
"Forward-Lasso Adaptive SHrinkage" (FLASH), which includes the Lasso and
Forward Selection as special cases, and can be used in both the linear
regression and the Generalized Linear Model domains. As with the Lasso and
Forward Selection, FLASH iteratively adds one variable to the model in a
hierarchical fashion but, unlike these methods, at each step adjusts the level
of shrinkage so as to optimize the selection of the next variable. We first
present FLASH in the linear regression setting and show that it can be fitted
using a variant of the computationally efficient LARS algorithm. Then, we
extend FLASH to the GLM domain and demonstrate, through numerous simulations
and real world data sets, as well as some theoretical analysis, that FLASH
generally outperforms many competing approaches.
