  Multi-hop question answering (QA) requires reasoning over multiple documents
to answer a complex question and provide interpretable supporting evidence.
However, providing supporting evidence is not enough to demonstrate that a
model has performed the desired reasoning to reach the correct answer. Most
existing multi-hop QA methods fail to answer a large fraction of sub-questions,
even if their parent questions are answered correctly. In this paper, we
propose the Prompt-based Conservation Learning (PCL) framework for multi-hop
QA, which acquires new knowledge from multi-hop QA tasks while conserving old
knowledge learned on single-hop QA tasks, mitigating forgetting. Specifically,
we first train a model on existing single-hop QA tasks, and then freeze this
model and expand it by allocating additional sub-networks for the multi-hop QA
task. Moreover, to condition pre-trained language models to stimulate the kind
of reasoning required for specific multi-hop questions, we learn soft prompts
for the novel sub-networks to perform type-specific reasoning. Experimental
results on the HotpotQA benchmark show that PCL is competitive for multi-hop QA
and retains good performance on the corresponding single-hop sub-questions,
demonstrating the efficacy of PCL in mitigating knowledge loss by forgetting.
