  Existing approaches to few-shot learning in NLP rely on large language models
and fine-tuning of these to generalise on out-of-distribution data. In this
work, we propose a simple yet powerful approach to "extreme" few-shot learning,
wherein models are exposed to as little as 4 examples per class, based on
soft-label prototypes that collectively capture the distribution of different
classes across the input domain space. Inspired by previous work (Sucholutsky
et al., 2021) on univariate or simple multivariate (synthetic) data, we propose
a novel approach that is effective on large, high-dimensional and real-world
datasets. We learn soft-label prototypes within a neural framework (DeepSLP)
and we experimentally demonstrate that it achieves superior performance on
31/48 tested tasks and few-shot settings while closely matching the performance
of strong baselines on the rest. We focus on learning previously unseen NLP
tasks from very few examples (4, 8, 16) per label and present an in-depth
analysis of the effectiveness of our approach.
