  This paper investigates biases of Large Language Models (LLMs) through the
lens of grammatical gender. Drawing inspiration from seminal works in
psycholinguistics, particularly the study of gender's influence on language
perception, we leverage multilingual LLMs to revisit and expand upon the
foundational experiments of Boroditsky (2003). Employing LLMs as a novel method
for examining psycholinguistic biases related to grammatical gender, we prompt
a model to describe nouns with adjectives in various languages, focusing
specifically on languages with grammatical gender. In particular, we look at
adjective co-occurrences across gender and languages, and train a binary
classifier to predict grammatical gender given adjectives an LLM uses to
describe a noun. Surprisingly, we find that a simple classifier can not only
predict noun gender above chance but also exhibit cross-language
transferability. We show that while LLMs may describe words differently in
different languages, they are biased similarly.
