  The goal of our research is to contribute information about how useful the
crowd is at anticipating stereotypes that may be biasing a data set without a
researcher's knowledge. The results of the crowd's prediction can potentially
be used during data collection to help prevent the suspected stereotypes from
introducing bias to the dataset. We conduct our research by asking the crowd on
Amazon's Mechanical Turk (AMT) to complete two similar Human Intelligence Tasks
(HITs) by suggesting stereotypes relating to their personal experience. Our
analysis of these responses focuses on determining the level of diversity in
the workers' suggestions and their demographics. Through this process we begin
a discussion on how useful the crowd can be in tackling this difficult problem
within machine learning data collection.
