  Multiple testing adjustments, such as the Benjamini and Hochberg (1995)
step-up procedure for controlling the false discovery rate (FDR), are typically
applied to families of tests that control significance level in the classical
sense: for each individual test, the probability of false rejection is no
greater than the nominal level. In this paper, we consider tests that satisfy
only a weaker notion of significance level control, in which the probability of
false rejection need only be controlled on average over the hypotheses. We find
that the Benjamini and Hochberg (1995) step-up procedure still controls FDR in
the asymptotic regime with many weakly dependent $p$-values, and that certain
adjustments for dependent $p$-values such as the Benjamini and Yekutieli (2001)
procedure continue to yield FDR control in finite samples. Our results open the
door to FDR controlling procedures in nonparametric and high dimensional
settings where weakening the notion of inference allows for large power
improvements.
