  Benchmarks are seen as the cornerstone for measuring technical progress in
Artificial Intelligence (AI) research and have been developed for a variety of
tasks ranging from question answering to facial recognition. An increasingly
prominent research area in AI is ethics, which currently has no set of
benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI
system. In this paper, drawing upon research in moral philosophy and
metaethics, we argue that it is impossible to develop such a benchmark. As
such, alternative mechanisms are necessary for evaluating whether an AI system
is 'ethical'. This is especially pressing in light of the prevalence of
applied, industrial AI research. We argue that it makes more sense to talk
about 'values' (and 'value alignment') rather than 'ethics' when considering
the possible actions of present and future AI systems. We further highlight
that, because values are unambiguously relative, focusing on values forces us
to consider explicitly what the values are and whose values they are. Shifting
the emphasis from ethics to values therefore gives rise to several new ways of
understanding how researchers might advance research programmes for robustly
safe or beneficial AI. We conclude by highlighting a number of possible ways
forward for the field as a whole, and we advocate for different approaches
towards more value-aligned AI research.
