  Question answering models commonly have access to two sources of "knowledge"
during inference time: (1) parametric knowledge - the factual knowledge encoded
in the model weights, and (2) contextual knowledge - external knowledge (e.g.,
a Wikipedia passage) given to the model to generate a grounded answer. Having
these two sources of knowledge entangled together is a core issue for
generative QA models as it is unclear whether the answer stems from the given
non-parametric knowledge or not. This unclarity has implications on issues of
trust, interpretability and factuality. In this work, we propose a new paradigm
in which QA models are trained to disentangle the two sources of knowledge.
Using counterfactual data augmentation, we introduce a model that predicts two
answers for a given question: one based on given contextual knowledge and one
based on parametric knowledge. Our experiments on the Natural Questions dataset
show that this approach improves the performance of QA models by making them
more robust to knowledge conflicts between the two knowledge sources, while
generating useful disentangled answers.
