  We study the problem of selecting the best $m$ units from a set of $n$ as $m
/ n \to \alpha \in (0, 1)$, where noisy, heteroskedastic measurements of the
units' true values are available and the decision-maker wishes to maximize the
average true value of the units selected. Given a parametric prior
distribution, the empirical Bayes decision rule incurs $O_p(n^{-1})$ regret
relative to the Bayesian oracle that knows the true prior. More generally, if
the error in the estimated prior is of order $O_p(r_n)$, regret is
$O_p(r_n^2)$. In this sense selecting the best units is easier than estimating
their values. We show this regret bound is sharp in the parametric case, by
giving an example in which it is attained. Using priors calibrated from a
dataset of over four thousand internet experiments, we find that empirical
Bayes methods perform well in practice for detecting the best treatments given
only a modest number of experiments.
