  Analyzing large-scale, multi-experiment studies requires scientists to test
each experimental outcome for statistical significance and then assess the
results as a whole. We present Black Box FDR (BB-FDR), an empirical-Bayes
method for analyzing multi-experiment studies when many covariates are gathered
per experiment. BB-FDR learns a series of black box predictive models to boost
power and control the false discovery rate (FDR) at two stages of study
analysis. In Stage 1, it uses a deep neural network prior to report which
experiments yielded significant outcomes. In Stage 2, a separate black box
model of each covariate is used to select features that have significant
predictive power across all experiments. In benchmarks, BB-FDR outperforms
competing state-of-the-art methods in both stages of analysis. We apply BB-FDR
to two real studies on cancer drug efficacy. For both studies, BB-FDR increases
the proportion of significant outcomes discovered and selects variables that
reveal key genomic drivers of drug sensitivity and resistance in cancer.
