  We consider two-class linear classification in a high-dimensional, low-sample
size setting. Only a small fraction of the features are useful, the useful
features are unknown to us, and each useful feature contributes weakly to the
classification decision -- this setting was called the rare/weak model (RW
Model). We select features by thresholding feature $z$-scores. The threshold is
set by {\it higher criticism} (HC). Let $\pee_i$ denote the $P$-value
associated to the $i$-th $z$-score and $\pee_{(i)}$ denote the $i$-th order
statistic of the collection of $P$-values. The HC threshold (HCT) is the order
statistic of the $z$-score corresponding to index $i$ maximizing $(i/n -
\pee_{(i)})/\sqrt{\pee_{(i)}(1-\pee_{(i)})}$. The ideal threshold optimizes the
classification error. In \cite{PNAS} we showed that HCT was numerically close
to the ideal threshold. We formalize an asymptotic framework for studying the
RW model, considering a sequence of problems with increasingly many features
and relatively fewer observations. We show that along this sequence, the
limiting performance of ideal HCT is essentially just as good as the limiting
performance of ideal thresholding. Our results describe two-dimensional {\it
phase space}, a two-dimensional diagram with coordinates quantifying "rare" and
"weak" in the RW model. Phase space can be partitioned into two regions -- one
where ideal threshold classification is successful, and one where the features
are so weak and so rare that it must fail. Surprisingly, the regions where
ideal HCT succeeds and fails make the exact same partition of the phase
diagram. Other threshold methods, such as FDR threshold selection, are
successful in a substantially smaller region of the phase space than either HCT
or Ideal thresholding.
