  Many high dimensional classification techniques have been proposed in the
literature based on sparse linear discriminant analysis (LDA). To efficiently
use them, sparsity of linear classifiers is a prerequisite. However, this might
not be readily available in many applications, and rotations of data are
required to create the needed sparsity. In this paper, we propose a family of
rotations to create the required sparsity. The basic idea is to use the
principal components of the sample covariance matrix of the pooled samples and
its variants to rotate the data first and to then apply an existing high
dimensional classifier. This rotate-and-solve procedure can be combined with
any existing classifiers, and is robust against the sparsity level of the true
model. We show that these rotations do create the sparsity needed for high
dimensional classifications and provide theoretical understanding why such a
rotation works empirically. The effectiveness of the proposed method is
demonstrated by a number of simulated and real data examples, and the
improvements of our method over some popular high dimensional classification
rules are clearly shown.
