  The pre-trained language model is trained on large-scale unlabeled text and
can achieve state-of-the-art results in many different downstream tasks.
However, the current pre-trained language model is mainly concentrated in the
Chinese and English fields. For low resource language such as Tibetan, there is
lack of a monolingual pre-trained model. To promote the development of Tibetan
natural language processing tasks, this paper collects the large-scale training
data from Tibetan websites and constructs a vocabulary that can cover 99.95$\%$
of the words in the corpus by using Sentencepiece. Then, we train the Tibetan
monolingual pre-trained language model named TiBERT on the data and vocabulary.
Finally, we apply TiBERT to the downstream tasks of text classification and
question generation, and compare it with classic models and multilingual
pre-trained models, the experimental results show that TiBERT can achieve the
best performance. Our model is published in http://tibert.cmli-nlp.com/
