  The paper describes neural models developed for the DAGPap22 shared task
hosted at the Third Workshop on Scholarly Document Processing. This shared task
targets the automatic detection of generated scientific papers. Our work
focuses on comparing different transformer-based models as well as using
additional datasets and techniques to deal with imbalanced classes. As a final
submission, we utilized an ensemble of SciBERT, RoBERTa, and DeBERTa fine-tuned
using random oversampling technique. Our model achieved 99.24% in terms of
F1-score. The official evaluation results have put our system at the third
place.
