  Large language models can perform new tasks in a zero-shot fashion, given
natural language prompts that specify the desired behavior. Such prompts are
typically hand engineered, but can also be learned with gradient-based methods
from labeled data. However, it is underexplored what factors make the prompts
effective, especially when the prompts are natural language. In this paper, we
investigate common attributes shared by effective prompts. We first propose a
human readable prompt tuning method (F LUENT P ROMPT) based on Langevin
dynamics that incorporates a fluency constraint to find a diverse distribution
of effective and fluent prompts. Our analysis reveals that effective prompts
are topically related to the task domain and calibrate the prior probability of
label words. Based on these findings, we also propose a method for generating
prompts using only unlabeled data, outperforming strong baselines by an average
of 7.0% accuracy across three tasks.
