  Inspired by the 'Bias Considerations in Bilingual Natural Language
Processing' report by Statistics Canada, this study delves into potential
biases in multilingual sentiment analysis between English and French. Given a
50-50 dataset of French and English, we aim to determine if there exists a
language bias and explore how the incorporation of more diverse datasets in the
future might affect the equity of multilingual Natural Language Processing
(NLP) systems. By employing Support Vector Machine (SVM) and Naive Bayes models
on three balanced datasets, we reveal potential biases in multilingual
sentiment classification. Utilizing Fairlearn, a tool for assessing bias in
machine learning models, our findings indicate nuanced outcomes. With French
data outperforming English across accuracy, recall, and F1 score metrics in
both models, hinting at a language bias favoring French. However, Fairlearn's
metrics suggest that the SVM approaches equitable levels with a demographic
parity ratio of 0.963, 0.989, and 0.985 for the three separate datasets,
indicating near-equitable treatment across languages. In contrast, Naive Bayes
demonstrates greater disparities, evidenced by a demographic parity ratio of
0.813, 0.908, and 0.961. These findings reveal the importance of developing
equitable multilingual NLP systems, particularly as we anticipate the inclusion
of more datasets in various languages in the future.
