  The surge of pre-trained language models has begun a new era in the field of
Natural Language Processing (NLP) by allowing us to build powerful language
models. Among these models, Transformer-based models such as BERT have become
increasingly popular due to their state-of-the-art performance. However, these
models are usually focused on English, leaving other languages to multilingual
models with limited resources. This paper proposes a monolingual BERT for the
Persian language (ParsBERT), which shows its state-of-the-art performance
compared to other architectures and multilingual models. Also, since the amount
of data available for NLP tasks in Persian is very restricted, a massive
dataset for different NLP tasks as well as pre-training the model is composed.
ParsBERT obtains higher scores in all datasets, including existing ones as well
as composed ones and improves the state-of-the-art performance by outperforming
both multilingual BERT and other prior works in Sentiment Analysis, Text
Classification and Named Entity Recognition tasks.
