  In this paper, a novel learning paradigm is presented to automatically
identify groups of informative and correlated features from very high
dimensions. Specifically, we explicitly incorporate correlation measures as
constraints and then propose an efficient embedded feature selection method
using recently developed cutting plane strategy. The benefits of the proposed
algorithm are two-folds. First, it can identify the optimal discriminative and
uncorrelated feature subset to the output labels, denoted here as Support
Features, which brings about significant improvements in prediction performance
over other state of the art feature selection methods considered in the paper.
Second, during the learning process, the underlying group structures of
correlated features associated with each support feature, denoted as Affiliated
Features, can also be discovered without any additional cost. These affiliated
features serve to improve the interpretations on the learning tasks. Extensive
empirical studies on both synthetic and very high dimensional real-world
datasets verify the validity and efficiency of the proposed method.
