  The "curse of dimensionality" is a well-known problem in pattern recognition.
A widely used approach to tackling the problem is a group of subspace methods,
where the original features are projected onto a new space. The lower
dimensional subspace is then used to approximate the original features for
classification. However, most subspace methods were not originally developed
for classification. We believe that direct adoption of these subspace methods
for pattern classification should not be considered best practice. In this
paper, we present a new information theory based algorithm for selecting
subspaces, which can always result in superior performance over conventional
methods. This paper makes the following main contributions: i) it improves a
common practice widely used by practitioners in the field of pattern
recognition, ii) it develops an information theory based technique for
systematically selecting the subspaces that are discriminative and therefore
are suitable for pattern recognition/classification purposes, iii) it presents
extensive experimental results on a variety of computer vision and pattern
recognition tasks to illustrate that the subspaces selected based on maximum
mutual information criterion will always enhance performance regardless of the
classification techniques used.
