  Prompt-based fine-tuning has boosted the performance of Pre-trained Language
Models (PLMs) on few-shot text classification by employing task-specific
prompts. Yet, PLMs are unfamiliar with prompt-style expressions during
pre-training, which limits the few-shot learning performance on downstream
tasks. It would be desirable if the models can acquire some prompting knowledge
before adaptation to specific NLP tasks. We present the Unified Prompt Tuning
(UPT) framework, leading to better few-shot text classification for BERT-style
models by explicitly capturing prompting semantics from non-target NLP
datasets. In UPT, a novel paradigm Prompt-Options-Verbalizer is proposed for
joint prompt learning across different NLP tasks, forcing PLMs to capture
task-invariant prompting knowledge. We further design a self-supervised task
named Knowledge-enhanced Selective Masked Language Modeling to improve the
PLM's generalization abilities for accurate adaptation to previously unseen
tasks. After multi-task learning across multiple tasks, the PLM can be better
prompt-tuned towards any dissimilar target tasks in low-resourced settings.
Experiments over a variety of NLP tasks show that UPT consistently outperforms
state-of-the-arts for prompt-based fine-tuning.
