  This paper presents a comprehensive examination of the impact of tokenization
strategies and vocabulary sizes on the performance of Arabic language models in
downstream natural language processing tasks. Our investigation focused on the
effectiveness of four tokenizers across various tasks, including News
Classification, Hate Speech Detection, Sentiment Analysis, and Natural Language
Inference. Leveraging a diverse set of vocabulary sizes, we scrutinize the
intricate interplay between tokenization approaches and model performance. The
results reveal that Byte Pair Encoding (BPE) with Farasa outperforms other
strategies in multiple tasks, underscoring the significance of morphological
analysis in capturing the nuances of the Arabic language. However, challenges
arise in sentiment analysis, where dialect specific segmentation issues impact
model efficiency. Computational efficiency analysis demonstrates the stability
of BPE with Farasa, suggesting its practical viability. Our study uncovers
limited impacts of vocabulary size on model performance while keeping the model
size unchanged. This is challenging the established beliefs about the
relationship between vocabulary, model size, and downstream tasks, emphasizing
the need for the study of models' size and their corresponding vocabulary size
to generalize across domains and mitigate biases, particularly in dialect based
datasets. Paper's recommendations include refining tokenization strategies to
address dialect challenges, enhancing model robustness across diverse
linguistic contexts, and expanding datasets to encompass the rich dialect based
Arabic. This work not only advances our understanding of Arabic language models
but also lays the foundation for responsible and ethical developments in
natural language processing technologies tailored to the intricacies of the
Arabic language.
