  Computer vision technology is being used by many but remains representative
of only a few. People have reported misbehavior of computer vision models,
including offensive prediction results and lower performance for
underrepresented groups. Current computer vision models are typically developed
using datasets consisting of manually annotated images or videos; the data and
label distributions in these datasets are critical to the models' behavior. In
this paper, we examine ImageNet, a large-scale ontology of images that has
spurred the development of many modern computer vision methods. We consider
three key factors within the "person" subtree of ImageNet that may lead to
problematic behavior in downstream computer vision technology: (1) the stagnant
concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of
all categories with images, and (3) the inequality of representation in the
images within concepts. We seek to illuminate the root causes of these concerns
and take the first steps to mitigate them constructively.
