  Feature selection has drawn much attention over the last decades in machine
learning because it can reduce data dimensionality while maintaining the
original physical meaning of features, which enables better interpretability
than feature extraction. However, most existing feature selection approaches,
especially deep-learning-based, often focus on the features with great
importance scores only but neglect those with less importance scores during
training as well as the order of important candidate features. This can be
risky since some important and relevant features might be unfortunately ignored
during training, leading to suboptimal solutions or misleading selections. In
our work, we deal with feature selection by exploiting the features with less
importance scores and propose a feature selection framework based on a novel
complementary feature mask. Our method is generic and can be easily integrated
into existing deep-learning-based feature selection approaches to improve their
performance as well. Experiments have been conducted on benchmarking datasets
and shown that the proposed method can select more representative and
informative features than the state of the art.
