  In this paper, we study the problem of learning the skill distribution of a
population of agents from observations of pairwise games in a tournament. These
games are played among randomly drawn agents from the population. The agents in
our model can be individuals, sports teams, or Wall Street fund managers.
Formally, we postulate that the likelihoods of game outcomes are governed by
the Bradley-Terry-Luce (or multinomial logit) model, where the probability of
an agent beating another is the ratio between its skill level and the pairwise
sum of skill levels, and the skill parameters are drawn from an unknown skill
density of interest. The problem is, in essence, to learn a distribution from
noisy, quantized observations. We propose a simple and tractable algorithm that
learns the skill density with near-optimal minimax mean squared error scaling
as $n^{-1+\varepsilon}$, for any $\varepsilon>0$, when the density is smooth.
Our approach brings together prior work on learning skill parameters from
pairwise comparisons with kernel density estimation from non-parametric
statistics. Furthermore, we prove minimax lower bounds which establish minimax
optimality of the skill parameter estimation technique used in our algorithm.
These bounds utilize a continuum version of Fano's method along with a covering
argument. We apply our algorithm to various soccer leagues and world cups,
cricket world cups, and mutual funds. We find that the entropy of a learnt
distribution provides a quantitative measure of skill, which provides rigorous
explanations for popular beliefs about perceived qualities of sporting events,
e.g., soccer league rankings. Finally, we apply our method to assess the skill
distributions of mutual funds. Our results shed light on the abundance of low
quality funds prior to the Great Recession of 2008, and the domination of the
industry by more skilled funds after the financial crisis.
