  The false discovery rate (FDR) and false nondiscovery rate (FNDR) have
received considerable attention in the literature on multiple testing. These
performance measures are also appropriate for classification, and in this work
we develop generalization error analyses for FDR and FNDR when learning a
classifier from labeled training data. Unlike more conventional classification
performance measures, the empirical FDR and FNDR are not binomial random
variables but rather a ratio of binomials, which introduces challenges not
addressed in conventional analyses. We develop distribution-free uniform
deviation bounds and apply these to obtain finite sample bounds and strong
universal consistency.
