  Prompting language models (LMs) is the main interface for applying them to
new tasks. However, for smaller LMs, prompting provides low accuracy compared
to gradient-based finetuning. Tree Prompting is an approach to prompting which
builds a decision tree of prompts, linking multiple LM calls together to solve
a task. At inference time, each call to the LM is determined by efficiently
routing the outcome of the previous call using the tree. Experiments on
classification datasets show that Tree Prompting improves accuracy over
competing methods and is competitive with fine-tuning. We also show that
variants of Tree Prompting allow inspection of a model's decision-making
process.
