  The traditional framework for feature selection treats all features as
costing the same amount. However, in reality, a scientist often has
considerable discretion regarding which variables to measure, and the decision
involves a tradeoff between model accuracy and cost (where cost can refer to
money, time, difficulty, or intrusiveness). In particular, unnecessarily
including an expensive feature in a model is worse than unnecessarily including
a cheap feature. We propose a procedure, which we call cheap knockoffs, for
performing feature selection in a cost-conscious manner. The key idea behind
our method is to force higher cost features to compete with more knockoffs than
cheaper features. We derive an upper bound on the weighted false discovery
proportion associated with this procedure, which corresponds to the fraction of
the feature cost that is wasted on unimportant features. We prove that this
bound holds simultaneously with high probability over a path of selected
variable sets of increasing size. A user may thus select a set of features
based, for example, on the overall budget, while knowing that no more than a
particular fraction of feature cost is wasted. We investigate, through
simulation and a biomedical application, the practical importance of
incorporating cost considerations into the feature selection process.
