  In many real world problems, features do not act alone but in combination
with each other. For example, in genomics, diseases might not be caused by any
single mutation but require the presence of multiple mutations. Prior work on
feature selection either seeks to identify individual features or can only
determine relevant groups from a predefined set. We investigate the problem of
discovering groups of predictive features without predefined grouping. To do
so, we define predictive groups in terms of linear and non-linear interactions
between features. We introduce a novel deep learning architecture that uses an
ensemble of feature selection models to find predictive groups, without
requiring candidate groups to be provided. The selected groups are sparse and
exhibit minimum overlap. Furthermore, we propose a new metric to measure
similarity between discovered groups and the ground truth. We demonstrate the
utility of our model on multiple synthetic tasks and semi-synthetic chemistry
datasets, where the ground truth structure is known, as well as an image
dataset and a real-world cancer dataset.
