  The rapid deployment of generative language models (LMs) has raised concerns
about social biases affecting the well-being of diverse consumers. The extant
literature on generative LMs has primarily examined bias via explicit identity
prompting. However, prior research on bias in earlier language-based technology
platforms, including search engines, has shown that discrimination can occur
even when identity terms are not specified explicitly. Studies of bias in LM
responses to open-ended prompts (where identity classifications are left
unspecified) are lacking and have not yet been grounded in end-consumer harms.
Here, we advance studies of generative LM bias by considering a broader set of
natural use cases via open-ended prompting. In this "laissez-faire" setting, we
find that synthetically generated texts from five of the most pervasive LMs
(ChatGPT3.5, ChatGPT4, Claude2.0, Llama2, and PaLM2) perpetuate harms of
omission, subordination, and stereotyping for minoritized individuals with
intersectional race, gender, and/or sexual orientation identities (AI/AN,
Asian, Black, Latine, MENA, NH/PI, Female, Non-binary, Queer). We find
widespread evidence of bias to an extent that such individuals are hundreds to
thousands of times more likely to encounter LM-generated outputs that portray
their identities in a subordinated manner compared to representative or
empowering portrayals. We also document a prevalence of stereotypes (e.g.
perpetual foreigner) in LM-generated outputs that are known to trigger
psychological harms that disproportionately affect minoritized individuals.
These include stereotype threat, which leads to impaired cognitive performance
and increased negative self-perception. Our findings highlight the urgent need
to protect consumers from discriminatory harms caused by language models and
invest in critical AI education programs tailored towards empowering diverse
consumers.
