  The usage of more than one language in the same text is referred to as Code
Mixed. It is evident that there is a growing degree of adaption of the use of
code-mixed data, especially English with a regional language, on social media
platforms. Existing deep-learning models do not take advantage of the implicit
language information in the code-mixed text. Our study aims to improve
BERT-based models performance on low-resource Code-Mixed Hindi-English Datasets
by experimenting with language augmentation approaches. We propose a pipeline
to improve code-mixed systems that comprise data preprocessing, word-level
language identification, language augmentation, and model training on
downstream tasks like sentiment analysis. For language augmentation in BERT
models, we explore word-level interleaving and post-sentence placement of
language information. We have examined the performance of vanilla BERT-based
models and their code-mixed HingBERT counterparts on respective benchmark
datasets, comparing their results with and without using word-level language
information. The models were evaluated using metrics such as accuracy,
precision, recall, and F1 score. Our findings show that the proposed language
augmentation approaches work well across different BERT models. We demonstrate
the importance of augmenting code-mixed text with language information on five
different code-mixed Hindi-English downstream datasets based on sentiment
analysis, hate speech detection, and emotion detection.
