  This paper introduces a study on tweet sentiment classification. Our task is
to classify a tweet as either positive or negative. We approach the problem in
two steps, namely embedding and classifying. Our baseline methods include
several combinations of traditional embedding methods and classification
algorithms. Furthermore, we explore the current state-of-the-art tweet analysis
model, BERTweet, and propose a novel approach in which features are engineered
from the hidden states and attention matrices of the model, inspired by
empirical study of the tweets. Using a multi-layer perceptron trained with a
high dropout rate for classification, our proposed approach achieves a
validation accuracy of 0.9111.
