  Named Entity Recognition (NER) systems play a vital role in NLP applications
such as machine translation, summarization, and question-answering. These
systems identify named entities, which encompass real-world concepts like
locations, persons, and organizations. Despite extensive research on NER
systems for the English language, they have not received adequate attention in
the context of low resource languages. In this work, we focus on NER for
low-resource language and present our case study in the context of the Indian
language Marathi. The advancement of NLP research revolves around the
utilization of pre-trained transformer models such as BERT for the development
of NER models. However, we focus on improving the performance of shallow models
based on CNN, and LSTM by combining the best of both worlds. In the era of
transformers, these traditional deep learning models are still relevant because
of their high computational efficiency. We propose a hybrid approach for
efficient NER by integrating a BERT-based subword tokenizer into vanilla
CNN/LSTM models. We show that this simple approach of replacing a traditional
word-based tokenizer with a BERT-tokenizer brings the accuracy of vanilla
single-layer models closer to that of deep pre-trained models like BERT. We
show the importance of using sub-word tokenization for NER and present our
study toward building efficient NLP systems. The evaluation is performed on
L3Cube-MahaNER dataset using tokenizers from MahaBERT, MahaGPT, IndicBERT, and
mBERT.
