  Citation function and citation sentiment are two essential aspects of
citation content analysis (CCA), which are useful for influence analysis, the
recommendation of scientific publications. However, existing studies are mostly
traditional machine learning methods, although deep learning techniques have
also been explored, the improvement of the performance seems not significant
due to insufficient training data, which brings difficulties to applications.
In this paper, we propose to fine-tune pre-trained contextual embeddings
ULMFiT, BERT, and XLNet for the task. Experiments on three public datasets show
that our strategy outperforms all the baselines in terms of the F1 score. For
citation function identification, the XLNet model achieves 87.2%, 86.90%, and
81.6% on DFKI, UMICH, and TKDE2019 datasets respectively, while it achieves
91.72% and 91.56% on DFKI and UMICH in term of citation sentiment
identification. Our method can be used to enhance the influence analysis of
scholars and scholarly publications.
