  We examine multi-task benchmarks in machine learning through the lens of
social choice theory. We draw an analogy between benchmarks and electoral
systems, where models are candidates and tasks are voters. This suggests a
distinction between cardinal and ordinal benchmark systems. The former
aggregate numerical scores into one model ranking; the latter aggregate
rankings for each task. We apply Arrow's impossibility theorem to ordinal
benchmarks to highlight the inherent limitations of ordinal systems,
particularly their sensitivity to the inclusion of irrelevant models. Inspired
by Arrow's theorem, we empirically demonstrate a strong trade-off between
diversity and sensitivity to irrelevant changes in existing multi-task
benchmarks. Our result is based on new quantitative measures of diversity and
sensitivity that we introduce. Sensitivity quantifies the impact that
irrelevant changes to tasks have on a benchmark. Diversity captures the degree
of disagreement in model rankings across tasks. We develop efficient
approximation algorithms for both measures, as exact computation is
computationally challenging. Through extensive experiments on seven cardinal
benchmarks and eleven ordinal benchmarks, we demonstrate a clear trade-off
between diversity and stability: The more diverse a multi-task benchmark, the
more sensitive to trivial changes it is. Additionally, we show that the
aggregated rankings of existing benchmarks are highly unstable under irrelevant
changes. The codes and data are available at
https://socialfoundations.github.io/benchbench/.
