  This paper presents a novel feature selection method based on the conditional
mutual information (CMI). The proposed High Order Conditional Mutual
Information Maximization (HOCMIM) incorporates high order dependencies into the
feature selection procedure and has a straightforward interpretation due to its
bottom-up derivation. The HOCMIM is derived from the CMI's chain expansion and
expressed as a maximization optimization problem. The maximization problem is
solved using a greedy search procedure, which speeds up the entire feature
selection process. The experiments are run on a set of benchmark datasets (20
in total). The HOCMIM is compared with eighteen state-of-the-art feature
selection algorithms, from the results of two supervised learning classifiers
(Support Vector Machine and K-Nearest Neighbor). The HOCMIM achieves the best
results in terms of accuracy and shows to be faster than high order feature
selection counterparts.
