  Author Name Disambiguation (AND) is the task of resolving which author
mentions in a bibliographic database refer to the same real-world person, and
is a critical ingredient of digital library applications such as search and
citation analysis. While many AND algorithms have been proposed, comparing them
is difficult because they often employ distinct features and are evaluated on
different datasets. In response to this challenge, we present S2AND, a unified
benchmark dataset for AND on scholarly papers, as well as an open-source
reference model implementation. Our dataset harmonizes eight disparate AND
datasets into a uniform format, with a single rich feature set drawn from the
Semantic Scholar (S2) database. Our evaluation suite for S2AND reports
performance split by facets like publication year and number of papers,
allowing researchers to track both global performance and measures of fairness
across facet values. Our experiments show that because previous datasets tend
to cover idiosyncratic and biased slices of the literature, algorithms trained
to perform well on one on them may generalize poorly to others. By contrast, we
show how training on a union of datasets in S2AND results in more robust models
that perform well even on datasets unseen in training. The resulting AND model
also substantially improves over the production algorithm in S2, reducing error
by over 50% in terms of $B^3$ F1. We release our unified dataset, model code,
trained models, and evaluation suite to the research community.
https://github.com/allenai/S2AND/
