  Social biases are encoded in word embeddings. This presents a unique
opportunity to study society historically and at scale, and a unique danger
when embeddings are used in downstream applications. Here, we investigate the
extent to which publicly-available word embeddings accurately reflect beliefs
about certain kinds of people as measured via traditional survey methods. We
find that biases found in word embeddings do, on average, closely mirror survey
data across seventeen dimensions of social meaning. However, we also find that
biases in embeddings are much more reflective of survey data for some
dimensions of meaning (e.g. gender) than others (e.g. race), and that we can be
highly confident that embedding-based measures reflect survey data only for the
most salient biases.
