  Machine reading comprehension is a heavily-studied research and test field
for evaluating new pre-trained language models (PrLMs) and fine-tuning
strategies, and recent studies have enriched the pre-trained language models
with syntactic, semantic and other linguistic information to improve the
performance of the models. In this paper, we imitate the human reading process
in connecting the anaphoric expressions and explicitly leverage the coreference
information of the entities to enhance the word embeddings from the pre-trained
language model, in order to highlight the coreference mentions of the entities
that must be identified for coreference-intensive question answering in QUOREF,
a relatively new dataset that is specifically designed to evaluate the
coreference-related performance of a model. We use two strategies to fine-tune
a pre-trained language model, namely, placing an additional encoder layer after
a pre-trained language model to focus on the coreference mentions or
constructing a relational graph convolutional network to model the coreference
relations. We demonstrate that the explicit incorporation of coreference
information in the fine-tuning stage performs better than the incorporation of
the coreference information in pre-training a language model.
