  We employ an audit design to investigate biases in state-of-the-art large
language models, including GPT-4. In our study, we prompt the models for advice
involving a named individual across a variety of scenarios, such as during car
purchase negotiations or election outcome predictions. We find that the advice
systematically disadvantages names that are commonly associated with racial
minorities and women. Names associated with Black women receive the least
advantageous outcomes. The biases are consistent across 42 prompt templates and
several models, indicating a systemic issue rather than isolated incidents.
While providing numerical, decision-relevant anchors in the prompt can
successfully counteract the biases, qualitative details have inconsistent
effects and may even increase disparities. Our findings underscore the
importance of conducting audits at the point of LLM deployment and
implementation to mitigate their potential for harm against marginalized
communities.
