  The effectiveness of the BERT model on multiple linguistic tasks has been
well documented. On the other hand, its potentials for narrow and specific
domains such as Legal, have not been fully explored. In this paper, we examine
how BERT can be used in the Arabic legal domain and try customizing this
language model for several downstream tasks using several different
domain-relevant training and testing datasets to train BERT from scratch. We
introduce the AraLegal-BERT, a bidirectional encoder Transformer-based model
that have been thoroughly tested and carefully optimized with the goal to
amplify the impact of NLP-driven solution concerning jurisprudence, legal
documents, and legal practice. We fine-tuned AraLegal-BERT and evaluated it
against three BERT variations for Arabic language in three natural languages
understanding (NLU) tasks. The results show that the base version of
AraLegal-BERT achieve better accuracy than the general and original BERT over
the Legal text.
