  Speculation is a naturally occurring phenomena in textual data, forming an
integral component of many systems, especially in the biomedical information
retrieval domain. Previous work addressing cue detection and scope resolution
(the two subtasks of speculation detection) have ranged from rule-based systems
to deep learning-based approaches. In this paper, we apply three popular
transformer-based architectures, BERT, XLNet and RoBERTa to this task, on two
publicly available datasets, BioScope Corpus and SFU Review Corpus, reporting
substantial improvements over previously reported results (by at least 0.29 F1
points on cue detection and 4.27 F1 points on scope resolution). We also
experiment with joint training of the model on multiple datasets, which
outperforms the single dataset training approach by a good margin. We observe
that XLNet consistently outperforms BERT and RoBERTa, contrary to results on
other benchmark datasets. To confirm this observation, we apply XLNet and
RoBERTa to negation detection and scope resolution, reporting state-of-the-art
results on negation scope resolution for the BioScope Corpus (increase of 3.16
F1 points on the BioScope Full Papers, 0.06 F1 points on the BioScope
Abstracts) and the SFU Review Corpus (increase of 0.3 F1 points).
