  Text classification is a typical natural language processing or computational
linguistics task with various interesting applications. As the number of users
on social media platforms increases, data acceleration promotes emerging
studies on Social Media Text Classification (SMTC) or social media text mining
on these valuable resources. In contrast to English, Vietnamese, one of the
low-resource languages, is still not concentrated on and exploited thoroughly.
Inspired by the success of the GLUE, we introduce the Social Media Text
Classification Evaluation (SMTCE) benchmark, as a collection of datasets and
models across a diverse set of SMTC tasks. With the proposed benchmark, we
implement and analyze the effectiveness of a variety of multilingual BERT-based
models (mBERT, XLM-R, and DistilmBERT) and monolingual BERT-based models
(PhoBERT, viBERT, vELECTRA, and viBERT4news) for tasks in the SMTCE benchmark.
Monolingual models outperform multilingual models and achieve state-of-the-art
results on all text classification tasks. It provides an objective assessment
of multilingual and monolingual BERT-based models on the benchmark, which will
benefit future studies about BERTology in the Vietnamese language.
