  A hallmark of modern large language models (LLMs) is their impressive general
zero-shot and few-shot abilities, often elicited through in-context learning
(ICL) via prompting. However, while highly coveted and being the most general,
zero-shot performances in LLMs are still typically weaker due to the lack of
guidance and the difficulty of applying existing automatic prompt design
methods in general tasks when ground-truth labels are unavailable. In this
study, we address this by presenting Universal Self-Adaptive Prompting (USP),
an automatic prompt design approach specifically tailored for zero-shot
learning (while compatible with few-shot). Requiring only a small amount of
unlabeled data and an inference-only LLM, USP is highly versatile: to achieve
universal prompting, USP categorizes a possible NLP task into one of the three
possible task types and then uses a corresponding selector to select the most
suitable queries and zero-shot model-generated responses as
pseudo-demonstrations, thereby generalizing ICL to the zero-shot setup in a
fully automated way. We evaluate USP with PaLM and PaLM 2 models and
demonstrate performances that are considerably stronger than standard zero-shot
baselines and often comparable to or even superior to few-shot baselines across
more than 40 natural language understanding, natural language generation, and
reasoning tasks.
