  Computer vision is widely deployed, has highly visible, society altering
applications, and documented problems with bias and representation. Datasets
are critical for benchmarking progress in fair computer vision, and often
employ broad racial categories as population groups for measuring group
fairness. Similarly, diversity is often measured in computer vision datasets by
ascribing and counting categorical race labels. However, racial categories are
ill-defined, unstable temporally and geographically, and have a problematic
history of scientific use. Although the racial categories used across datasets
are superficially similar, the complexity of human race perception suggests the
racial system encoded by one dataset may be substantially inconsistent with
another. Using the insight that a classifier can learn the racial system
encoded by a dataset, we conduct an empirical study of computer vision datasets
supplying categorical race labels for face images to determine the
cross-dataset consistency and generalization of racial categories. We find that
each dataset encodes a substantially unique racial system, despite nominally
equivalent racial categories, and some racial categories are systemically less
consistent than others across datasets. We find evidence that racial categories
encode stereotypes, and exclude ethnic groups from categories on the basis of
nonconformity to stereotypes. Representing a billion humans under one racial
category may obscure disparities and create new ones by encoding stereotypes of
racial systems. The difficulty of adequately converting the abstract concept of
race into a tool for measuring fairness underscores the need for a method more
flexible and culturally aware than racial categories.
