  We introduce a novel class of variable selection penalties called TWIN, which
provides sensible data-adaptive penalization. Under a linear sparsity regime
and random Gaussian designs we show that penalties in the TWIN class have a
high probability of selecting the correct model and furthermore result in
minimax optimal estimators. The general shape of penalty functions in the TWIN
class is the key ingredient to its desirable properties and results in improved
theoretical and empirical performance over existing penalties. In this work we
introduce two examples of TWIN penalties that admit simple and efficient
coordinate descent algorithms, making TWIN practical in large data settings. We
demonstrate in challenging and realistic simulation settings with high
correlations between active and inactive variables that TWIN has high power in
variable selection while controlling the number of false discoveries,
outperforming standard penalties.
