  This paper presents several BERT-based models for Russian language biomedical
text mining (RuBioBERT, RuBioRoBERTa). The models are pre-trained on a corpus
of freely available texts in the Russian biomedical domain. With this
pre-training, our models demonstrate state-of-the-art results on RuMedBench -
Russian medical language understanding benchmark that covers a diverse set of
tasks, including text classification, question answering, natural language
inference, and named entity recognition.
