  Multilingual language models have gained significant attention in recent
years, enabling the development of applications that meet diverse linguistic
contexts. In this paper, we present a comprehensive evaluation of three popular
multilingual language models: mBERT, XLM-R, and GPT-3. We assess their
performance across a diverse set of languages, with a focus on understanding
the impact of resource availability (general and model-specific), language
family, script type, and word order on model performance, under two distinct
tasks - text classification and text generation. Our findings reveal that while
the amount of language-specific pretraining data plays a crucial role in model
performance, we also identify other factors such as general resource
availability, language family, and script type, as important features. We hope
that our study contributes to a deeper understanding of multilingual language
models to enhance their performance across languages and linguistic contexts.
