  Ethical reasoning is a crucial skill for Large Language Models (LLMs).
However, moral values are not universal, but rather influenced by language and
culture. This paper explores how three prominent LLMs -- GPT-4, ChatGPT, and
Llama2-70B-Chat -- perform ethical reasoning in different languages and if
their moral judgement depend on the language in which they are prompted. We
extend the study of ethical reasoning of LLMs by Rao et al. (2023) to a
multilingual setup following their framework of probing LLMs with ethical
dilemmas and policies from three branches of normative ethics: deontology,
virtue, and consequentialism. We experiment with six languages: English,
Spanish, Russian, Chinese, Hindi, and Swahili. We find that GPT-4 is the most
consistent and unbiased ethical reasoner across languages, while ChatGPT and
Llama2-70B-Chat show significant moral value bias when we move to languages
other than English. Interestingly, the nature of this bias significantly vary
across languages for all LLMs, including GPT-4.
