  Model-X knockoffs allows analysts to perform feature selection using almost
any machine learning algorithm while still provably controlling the expected
proportion of false discoveries. To apply model-X knockoffs, one must construct
synthetic variables, called knockoffs, which effectively act as controls during
feature selection. The gold standard for constructing knockoffs has been to
minimize the mean absolute correlation (MAC) between features and their
knockoffs, but, surprisingly, we prove this procedure can be powerless in
extremely easy settings, including Gaussian linear models with correlated
exchangeable features. The key problem is that minimizing the MAC creates
strong joint dependencies between the features and knockoffs, which allow
machine learning algorithms to partially or fully reconstruct the effect of the
features on the response using the knockoffs. To improve the power of
knockoffs, we propose generating knockoffs which minimize the
reconstructability (MRC) of the features, and we demonstrate our proposal for
Gaussian features by showing it is computationally efficient, robust, and
powerful. We also prove that certain MRC knockoffs minimize a natural
definition of estimation error in Gaussian linear models. Furthermore, in an
extensive set of simulations, we find many settings with correlated features in
which MRC knockoffs dramatically outperform MAC-minimizing knockoffs and no
settings in which MAC-minimizing knockoffs outperform MRC knockoffs by more
than a very slight margin. We implement our methods and a host of others from
the knockoffs literature in a new open source python package knockpy.
