  Large Language Models (LLMs) have recently emerged as an effective tool to
assist individuals in writing various types of content, including professional
documents such as recommendation letters. Though bringing convenience, this
application also introduces unprecedented fairness concerns. Model-generated
reference letters might be directly used by users in professional scenarios. If
underlying biases exist in these model-constructed letters, using them without
scrutinization could lead to direct societal harms, such as sabotaging
application success rates for female applicants. In light of this pressing
issue, it is imminent and necessary to comprehensively study fairness issues
and associated harms in this real-world use case. In this paper, we critically
examine gender biases in LLM-generated reference letters. Drawing inspiration
from social science findings, we design evaluation methods to manifest biases
through 2 dimensions: (1) biases in language style and (2) biases in lexical
content. We further investigate the extent of bias propagation by analyzing the
hallucination bias of models, a term that we define to be bias exacerbation in
model-hallucinated contents. Through benchmarking evaluation on 2 popular LLMs-
ChatGPT and Alpaca, we reveal significant gender biases in LLM-generated
recommendation letters. Our findings not only warn against using LLMs for this
application without scrutinization, but also illuminate the importance of
thoroughly studying hidden biases and harms in LLM-generated professional
documents.
