  Feature selection is an essential problem in computer vision, important for
category learning and recognition. Along with the rapid development of a wide
variety of visual features and classifiers, there is a growing need for
efficient feature selection and combination methods, to construct powerful
classifiers for more complex and higher-level recognition tasks. We propose an
algorithm that efficiently discovers sparse, compact representations of input
features or classifiers, from a vast sea of candidates, with important
optimality properties, low computational cost and excellent accuracy in
practice. Different from boosting, we start with a discriminant linear
classification formulation that encourages sparse solutions. Then we obtain an
equivalent unsupervised clustering problem that jointly discovers ensembles of
diverse features. They are independently valuable but even more powerful when
united in a cluster of classifiers. We evaluate our method on the task of
large-scale recognition in video and show that it significantly outperforms
classical selection approaches, such as AdaBoost and greedy forward-backward
selection, and powerful classifiers such as SVMs, in speed of training and
performance, especially in the case of limited training data.
