  Large language models (LLMs) have shown impressive capabilities in real-world
applications. The capability of in-context learning (ICL) allows us to adapt an
LLM to downstream tasks by including input-label exemplars in the prompt
without model fine-tuning. However, the quality of these exemplars in the
prompt greatly impacts performance, highlighting the need for an effective
automated exemplar selection method. Recent studies have explored
retrieval-based approaches to select exemplars tailored to individual test
queries, which can be undesirable due to extra test-time computation and an
increased risk of data exposure. Moreover, existing methods fail to adequately
account for the impact of exemplar ordering on the performance. On the other
hand, the impact of the instruction, another essential component in the prompt
given to the LLM, is often overlooked in existing exemplar selection methods.
To address these challenges, we propose a novel method named EASE, which
leverages the hidden embedding from a pre-trained language model to represent
ordered sets of exemplars and uses a neural bandit algorithm to optimize the
sets of exemplars while accounting for exemplar ordering. Our EASE can
efficiently find an ordered set of exemplars that performs well for all test
queries from a given task, thereby eliminating test-time computation.
Importantly, EASE can be readily extended to jointly optimize both the
exemplars and the instruction. Through extensive empirical evaluations
(including novel tasks), we demonstrate the superiority of EASE over existing
methods, and reveal practical insights about the impact of exemplar selection
on ICL, which may be of independent interest. Our code is available at
https://github.com/ZhaoxuanWu/EASE-Prompt-Optimization.
