  Feature selection is among the most important components because it not only
helps enhance the classification accuracy, but also or even more important
provides potential biomarker discovery. However, traditional multivariate
methods is likely to obtain unstable and unreliable results in case of an
extremely high dimensional feature space and very limited training samples,
where the features are often correlated or redundant. In order to improve the
stability, generalization and interpretations of the discovered potential
biomarker and enhance the robustness of the resultant classifier, the redundant
but informative features need to be also selected. Therefore we introduced a
novel feature selection method which combines a recent implementation of the
stability selection approach and the elastic net approach. The advantage in
terms of better control of false discoveries and missed discoveries of our
approach, and the resulted better interpretability of the obtained potential
biomarker is verified in both synthetic and real fMRI experiments. In addition,
we are among the first to demonstrate the robustness of feature selection
benefiting from the incorporation of stability selection and also among the
first to demonstrate the possible unrobustness of the classical univariate
two-sample t-test method. Specifically, we show the robustness of our feature
selection results in existence of noisy (wrong) training labels, as well as the
robustness of the resulted classifier based on our feature selection results in
the existence of data variation, demonstrated by a multi-center
attention-deficit/hyperactivity disorder (ADHD) fMRI data.
