  We explore and illustrate the concept of ranked sparsity, a phenomenon that
often occurs naturally in modeling applications when an expected disparity
exists in the quality of information between different feature sets. Its
presence can cause traditional and modern model selection methods to fail
because such procedures commonly presume that each potential parameter is
equally worthy of entering into the final model - we call this presumption
"covariate equipoise". However, this presumption does not always hold,
especially in the presence of derived variables. For instance, when all
possible interactions are considered as candidate predictors, the premise of
covariate equipoise will often produce over-specified and opaque models. The
sheer number of additional candidate variables grossly inflates the number of
false discoveries in the interactions, resulting in unnecessarily complex and
difficult-to-interpret models with many (truly spurious) interactions. We
suggest a modeling strategy that requires a stronger level of evidence in order
to allow certain variables (e.g. interactions) to be selected in the final
model. This ranked sparsity paradigm can be implemented with the
sparsity-ranked lasso (SRL). We compare the performance of SRL relative to
competing methods in a series of simulation studies, showing that the SRL is a
very attractive method because it is fast, accurate, and produces more
transparent models (with fewer false interactions). We illustrate its utility
in an application to predict the survival of lung cancer patients using a set
of gene expression measurements and clinical covariates, searching in
particular for gene-environment interactions.
