  Feature selection, which searches for the most representative features in
observed data, is critical for health data analysis. Unlike feature extraction,
such as PCA and autoencoder based methods, feature selection preserves
interpretability, meaning that the selected features provide direct information
about certain health conditions (i.e., the label). Thus, feature selection
allows domain experts, such as clinicians, to understand the predictions made
by machine learning based systems, as well as improve their own diagnostic
skills. Mutual information is often used as a basis for feature selection since
it measures dependencies between features and labels. In this paper, we
introduce a novel mutual information based feature selection (MIBFS) method
called SURI, which boosts features with high unique relevant information. We
compare SURI to existing MIBFS methods using 3 different classifiers on 6
publicly available healthcare data sets. The results indicate that, in addition
to preserving interpretability, SURI selects more relevant feature subsets
which lead to higher classification performance. More importantly, we explore
the dynamics of mutual information on a public low-dimensional health data set
via exhaustive search. The results suggest the important role of unique
relevant information in feature selection and verify the principles behind
SURI.
