  Prompt engineering is a challenging yet crucial task for optimizing the
performance of large language models on customized tasks. It requires complex
reasoning to examine the model's errors, hypothesize what is missing or
misleading in the current prompt, and communicate the task with clarity. While
recent works indicate that large language models can be meta-prompted to
perform automatic prompt engineering, we argue that their potential is limited
due to insufficient guidance for complex reasoning in the meta-prompt. We fill
this gap by infusing into the meta-prompt three key components: detailed
descriptions, context specification, and a step-by-step reasoning template. The
resulting method, named PE2, exhibits remarkable versatility across diverse
language tasks. It finds prompts that outperform "let's think step by step" by
6.3% on MultiArith and 3.1% on GSM8K, and outperforms competitive baselines on
counterfactual tasks by 6.9%. Further, we show that PE2 can make targeted and
highly specific prompt edits, rectify erroneous prompts, and induce multi-step
plans for complex tasks.
