  Pretrained Language Models (PLMs) have achieved tremendous success in natural
language understanding tasks. While different learning schemes -- fine-tuning,
zero-shot, and few-shot learning -- have been widely explored and compared for
languages such as English, there is comparatively little work in Chinese to
fairly and comprehensively evaluate and compare these methods and thus hinders
cumulative progress. In this paper, we introduce the Chinese Few-shot Learning
Evaluation Benchmark (FewCLUE), the first comprehensive few-shot evaluation
benchmark in Chinese. It includes nine tasks, ranging from single-sentence and
sentence-pair classification tasks to machine reading comprehension tasks. We
systematically evaluate five state-of-the-art (SOTA) few-shot learning methods
(including PET, ADAPET, LM-BFF, P-tuning and EFL), and compare their
performance with fine-tuning and zero-shot learning schemes on the newly
constructed FewCLUE benchmark. Experimental results reveal that: 1) The effect
of different few-shot learning methods is sensitive to the pre-trained model to
which the methods are applied; 2) PET and P-tuning achieve the best overall
performance with RoBERTa and ERNIE respectively. Our benchmark is used in the
few-shot learning contest of NLPCC 2021. In addition, we provide a
user-friendly toolkit, as well as an online leaderboard to help facilitate
further progress on Chinese few-shot learning. We provide a baseline
performance on different learning methods, a reference for future research.
