  There have been growing concerns regarding the out-of-domain generalization
ability of natural language processing (NLP) models, particularly in
question-answering (QA) tasks. Current synthesized data augmentation methods
for QA are hampered by increased training costs. To address this issue, we
propose a novel approach that combines prompting methods and linear probing
then fine-tuning strategy, which does not entail additional cost. Our method
has been theoretically and empirically shown to be effective in enhancing the
generalization ability of both generative and discriminative models. Our
approach outperforms state-of-the-art baselines, with an average increase in F1
score of 4.5%-7.9%. Furthermore, our method can be easily integrated into any
pre-trained models and offers a promising solution to the under-explored
cross-domain QA task. We release our source code at GitHub*.
