  Prompting pre-trained language models leads to promising results across
natural language processing tasks but is less effective when applied in
low-resource domains, due to the domain gap between the pre-training data and
the downstream task. In this work, we bridge this gap with a novel and
lightweight prompting methodology called SwitchPrompt for the adaptation of
language models trained on datasets from the general domain to diverse
low-resource domains. Using domain-specific keywords with a trainable gated
prompt, SwitchPrompt offers domain-oriented prompting, that is, effective
guidance on the target domains for general-domain language models. Our few-shot
experiments on three text classification benchmarks demonstrate the efficacy of
the general-domain pre-trained language models when used with SwitchPrompt.
They often even outperform their domain-specific counterparts trained with
baseline state-of-the-art prompting methods by up to 10.7% performance increase
in accuracy. This result indicates that SwitchPrompt effectively reduces the
need for domain-specific language model pre-training.
