  When dealing with text data containing subjective labels like speaker
emotions, inaccuracies or discrepancies among labelers are not uncommon. Such
discrepancies can significantly affect the performance of machine learning
algorithms. This study investigates the potential of identifying and addressing
outliers in text data with subjective labels, aiming to enhance classification
outcomes. We utilized the Deep SVDD algorithm, a one-class classification
method, to detect outliers in nine text-based emotion and sentiment analysis
datasets. By employing both a small-sized language model (DistilBERT base model
with 66 million parameters) and non-deep learning machine learning algorithms
(decision tree, KNN, Logistic Regression, and LDA) as the classifier, our
findings suggest that the removal of outliers can lead to enhanced results in
most cases. Additionally, as outliers in such datasets are not necessarily
unlearnable, we experienced utilizing a large language model -- DeBERTa v3
large with 131 million parameters, which can capture very complex patterns in
data. We continued to observe performance enhancements across multiple
datasets.
