  Text-to-image models are now easy to use and ubiquitous. However, prior work
has found that they are prone to recapitulating harmful Western stereotypes.
For example, requesting that a model generate an "African person and their
house," may produce a person standing next to a straw hut. In this example, the
word "African" is an explicit descriptor of the person that the prompt is
seeking to depict. Here, we examine whether implicit markers, such as dialect,
can also affect the portrayal of people in text-to-image outputs. We pair
prompts in Mainstream American English with counterfactuals that express
grammatical constructions found in dialects correlated with historically
marginalized groups. We find that through minimal, syntax-only changes to
prompts, we can systematically shift the skin tone and gender of people in the
generated images. We conclude with a discussion of whether dialectic
distribution shifts like this are harmful or are expected, possibly even
desirable, model behavior.
