  Malaysian English is a low resource creole language, where it carries the
elements of Malay, Chinese, and Tamil languages, in addition to Standard
English. Named Entity Recognition (NER) models underperform when capturing
entities from Malaysian English text due to its distinctive morphosyntactic
adaptations, semantic features and code-switching (mixing English and Malay).
Considering these gaps, we introduce MENmBERT and MENBERT, a pre-trained
language model with contextual understanding, specifically tailored for
Malaysian English. We have fine-tuned MENmBERT and MENBERT using manually
annotated entities and relations from the Malaysian English News Article (MEN)
Dataset. This fine-tuning process allows the PLM to learn representations that
capture the nuances of Malaysian English relevant for NER and RE tasks.
MENmBERT achieved a 1.52\% and 26.27\% improvement on NER and RE tasks
respectively compared to the bert-base-multilingual-cased model. Although the
overall performance of NER does not have a significant improvement, our further
analysis shows that there is a significant improvement when evaluated by the 12
entity labels. These findings suggest that pre-training language models on
language-specific and geographically-focused corpora can be a promising
approach for improving NER performance in low-resource settings. The dataset
and code published in this paper provide valuable resources for NLP research
work focusing on Malaysian English.
