  The term "Code Mixed" refers to the use of more than one language in the same
text. This phenomenon is predominantly observed on social media platforms, with
an increasing amount of adaptation as time goes on. It is critical to detect
foreign elements in a language and process them correctly, as a considerable
number of individuals are using code-mixed languages that could not be
comprehended by understanding one of those languages. In this work, we focus on
low-resource Hindi-English code-mixed language and enhancing the performance of
different code-mixed natural language processing tasks such as sentiment
analysis, emotion recognition, and hate speech identification. We perform a
comparative analysis of different Transformer-based language Models pre-trained
using unsupervised approaches. We have included the code-mixed models like
HingBERT, HingRoBERTa, HingRoBERTa-Mixed, mBERT, and non-code-mixed models like
AlBERT, BERT, and RoBERTa for comparative analysis of code-mixed Hindi-English
downstream tasks. We report state-of-the-art results on respective datasets
using HingBERT-based models which are specifically pre-trained on real
code-mixed text. Our HingBERT-based models provide significant improvements
thus highlighting the poor performance of vanilla BERT models on code-mixed
text.
