  The ranking problem is to order a collection of units by some unobserved
parameter, based on observations from the associated distribution. This problem
arises naturally in a number of contexts, such as business, where we may want
to rank potential projects by profitability; or science, where we may want to
rank predictors potentially associated with some trait by the strength of the
association. This approach provides a valuable alternative to the sparsity
framework often used with big data. Most approaches to this problem are
empirical Bayesian, where we use the data to estimate the hyperparameters of
the prior distribution, then use that distribution to estimate the unobserved
parameter values. There are a number of different approaches to this problem,
based on different loss functions for mis-ranking units. Despite the number of
papers developing methods for this problem, there is no work on the consistency
of these methods. In this paper, we develop a general framework for consistency
of empirical Bayesian ranking methods, which includes nearly all commonly used
methods. We then determine conditions under which consistency holds. Given that
little work has been done on selection of prior distribution, and that the loss
functions developed are not strongly motivated, we consider the case where both
of these are misspecified. We show that provided the loss function is
reasonable; the prior distribution is not too light-tailed; and the error in
measuring each unit converges to zero at a fast enough rate compared with the
number of units (which is assumed to increase to infinity); all ranking methods
are consistent.
