  Researchers must stay current in their fields by regularly reviewing academic
literature, a task complicated by the daily publication of thousands of papers.
Traditional multi-label text classification methods often ignore semantic
relationships and fail to address the inherent class imbalances. This paper
introduces a novel approach using the SciBERT model and CNNs to systematically
categorize academic abstracts from the Elsevier OA CC-BY corpus. We use a
multi-segment input strategy that processes abstracts, body text, titles, and
keywords obtained via BERT topic modeling through SciBERT. Here, the [CLS]
token embeddings capture the contextual representation of each segment,
concatenated and processed through a CNN. The CNN uses convolution and pooling
to enhance feature extraction and reduce dimensionality, optimizing the data
for classification. Additionally, we incorporate class weights based on label
frequency to address the class imbalance, significantly improving the
classification F1 score and enhancing text classification systems and
literature review efficiency.
