  The monolingual Hindi BERT models currently available on the model hub do not
perform better than the multi-lingual models on downstream tasks. We present
L3Cube-HindBERT, a Hindi BERT model pre-trained on Hindi monolingual corpus.
Further, since Indic languages, Hindi and Marathi share the Devanagari script,
we train a single model for both languages. We release DevBERT, a Devanagari
BERT model trained on both Marathi and Hindi monolingual datasets. We evaluate
these models on downstream Hindi and Marathi text classification and named
entity recognition tasks. The HindBERT and DevBERT-based models show
significant improvements over multi-lingual MuRIL, IndicBERT, and XLM-R. Based
on these observations we also release monolingual BERT models for other Indic
languages Kannada, Telugu, Malayalam, Tamil, Gujarati, Assamese, Odia, Bengali,
and Punjabi. These models are shared at https://huggingface.co/l3cube-pune .
