  Understanding how racial information impacts human decision making in online
systems is critical in today's world. Prior work revealed that race information
of criminal defendants, when presented as a text field, had no significant
impact on users' judgements of recidivism. We replicated and extended this work
to explore how and when race information influences users' judgements, with
respect to the saliency of presentation. Our results showed that adding photos
to the race labels had a significant impact on recidivism predictions for users
who identified as female, but not for those who identified as male. The race of
the defendant also impacted these results, with black defendants being less
likely to be predicted to recidivate compared to white defendants. These
results have strong implications for how system-designers choose to display
race information, and cautions researchers to be aware of gender and race
effects when using Amazon Mechanical Turk workers.
