  Language models are known to encode a great amount of factual knowledge
through pretraining. However, such knowledge might be insufficient to cater to
user requests, requiring the model to integrate external knowledge sources and
adhere to user-provided specifications. When answering questions about ongoing
events, the model should use recent news articles to update its response; when
asked to provide recommendations, the model should prioritize user
specifications over retrieved product reviews; when some facts are edited in
the model, the updated facts should override all prior knowledge learned by the
model even if they are conflicting. In all of the cases above, the model faces
a decision between its own parametric knowledge, (retrieved) contextual
knowledge, and user instruction knowledge. In this paper, we (1) unify such
settings into the problem of knowledge preference and define a three-level
preference hierarchy over these knowledge sources; (2) compile a collection of
existing datasets IfQA, MQuAKE, and MRQA covering a combination of settings
(with/without user specifications, with/without context documents) to
systematically evaluate how well models obey the intended knowledge preference;
and (3) propose a dataset synthesis method that composes diverse
question-answer pairs with user assumptions and related context to directly
fine-tune LMs for instilling the hierarchy of knowledge. We demonstrate that a
7B model, fine-tuned on only a few thousand examples automatically generated by
our proposed method, effectively achieves superior performance (more than 18%
improvement across all evaluation benchmarks) in adhering to the desired
knowledge preference hierarchy.
