  Large neural language models are steadily contributing state-of-the-art
performance to question answering and other natural language and information
processing tasks. These models are expensive to train. We propose to evaluate
whether such pre-trained models can benefit from the addition of explicit
linguistics information without requiring retraining from scratch.
  We present a linguistics-informed question answering approach that extends
and fine-tunes a pre-trained transformer-based neural language model with
symbolic knowledge encoded with a heterogeneous graph transformer. We
illustrate the approach by the addition of syntactic information in the form of
dependency and constituency graphic structures connecting tokens and virtual
vertices.
  A comparative empirical performance evaluation with BERT as its baseline and
with Stanford Question Answering Dataset demonstrates the competitiveness of
the proposed approach. We argue, in conclusion and in the light of further
results of preliminary experiments, that the approach is extensible to further
linguistics information including semantics and pragmatics.
