  Recent work has shown how to prompt large language models with explanations
to obtain strong performance on textual reasoning tasks, i.e., the
chain-of-thought paradigm. However, subtly different explanations can yield
widely varying downstream task accuracy. Explanations that have not been
"tuned" for a task, such as off-the-shelf explanations written by nonexperts,
may lead to mediocre performance. This paper tackles the problem of how to
optimize explanation-infused prompts in a blackbox fashion. We first generate
sets of candidate explanations for each example in the prompt using a
leave-one-out scheme, then find an effective combination of these explanations
with a two-stage framework. We first evaluate explanations for each in-context
example in isolation according to two proxy metrics, log likelihood and
accuracy on new examples. Then, we search over combinations of explanations to
find one that yields high performance against a silver-labeled development set.
Across four textual reasoning tasks spanning question answering, mathematical
reasoning, and natural language inference, results show that our proxy metrics
correlate with ground truth accuracy and our overall method can effectively
improve prompts over crowdworker annotations and naive search strategies
