  Unsupervised Machine Learning techniques have been applied to Natural
Language Processing tasks and surpasses the benchmarks such as GLUE with great
success. Building language models approach achieves good results in one
language and it can be applied to multiple NLP task such as classification,
summarization, generation and etc as an out of box model. Among all the of the
classical approaches used in NLP, the masked language modeling is the most
used. In general, the only requirement to build a language model is presence of
the large corpus of textual data. Text classification engines uses a variety of
models from classical and state of art transformer models to classify texts for
in order to save costs. Sequence Classifiers are mostly used in the domain of
text classification. However Token classifiers also are viable candidate models
as well. Sequence Classifiers and Token Classifier both tend to improve the
classification predictions due to the capturing the context information
differently. This work aims to compare the performance of Sequence Classifier
and Token Classifiers and evaluate each model on the same set of data. In this
work, we are using a pre-trained model as the base model and Token Classifier
and Sequence Classier heads results of these two scoring paradigms with be
compared..
