  Language agency is an important aspect of evaluating social biases in texts.
While several studies approached agency-related bias in human-written language,
very limited research has investigated such biases in Large Language Model
(LLM)-generated content. In addition, previous research often relies on
string-matching techniques to identify agentic and communal words within texts,
which fall short of accurately classifying language agency. We introduce the
novel Language Agency Bias Evaluation (LABE) benchmark, which comprehensively
evaluates biases in LLMs by analyzing agency levels attributed to different
demographic groups in model generations. LABE leverages 5,400 template-based
prompts, an accurate agency classifier, and corresponding bias metrics to test
for gender, racial, and intersectional language agency biases in LLMs on 3 text
generation tasks: biographies, professor reviews, and reference letters. To
build better and more accurate automated agency classifiers, we also contribute
and release the Language Agency Classification (LAC) dataset, consisting of
3,724 agentic and communal sentences. Using LABE, we unveil previously
under-explored language agency social biases in 3 recent LLMs: ChatGPT, Llama3,
and Mistral. We observe that: (1) For the same text category, LLM generations
demonstrate higher levels of gender bias than human-written texts; (2) On most
generation tasks, models show remarkably higher levels of intersectional bias
than the other bias aspects. Those who are at the intersection of gender and
racial minority groups -- such as Black females -- are consistently described
by texts with lower levels of agency; (3) Among the 3 LLMs investigated, Llama3
demonstrates greatest overall bias in language agency; (4) Not only does
prompt-based mitigation fail to resolve language agency bias in LLMs, but it
frequently leads to the exacerbation of biases in generated texts.
