  While Large Language Models (LLM) have created a massive technological impact
in the past decade, allowing for human-enabled applications, they can produce
output that contains stereotypes and biases, especially when using low-resource
languages. This can be of great ethical concern when dealing with sensitive
topics such as religion. As a means toward making LLMS more fair, we explore
bias from a religious perspective in Bengali, focusing specifically on two main
religious dialects: Hindu and Muslim-majority dialects. Here, we perform
different experiments and audit showing the comparative analysis of different
sentences using three commonly used LLMs: ChatGPT, Gemini, and Microsoft
Copilot, pertaining to the Hindu and Muslim dialects of specific words and
showcasing which ones catch the social biases and which do not. Furthermore, we
analyze our findings and relate them to potential reasons and evaluation
perspectives, considering their global impact with over 300 million speakers
worldwide. With this work, we hope to establish the rigor for creating more
fairness in LLMs, as these are widely used as creative writing agents.
