  We propose a new method, Adversarial In-Context Learning (adv-ICL), to
optimize prompt for in-context learning (ICL) by employing one LLM as a
generator, another as a discriminator, and a third as a prompt modifier. As in
traditional adversarial learning, adv-ICL is implemented as a two-player game
between the generator and discriminator, where the generator tries to generate
realistic enough output to fool the discriminator. In each round, given an
input prefixed by task instructions and several exemplars, the generator
produces an output. The discriminator is then tasked with classifying the
generator input-output pair as model-generated or real data. Based on the
discriminator loss, the prompt modifier proposes possible edits to the
generator and discriminator prompts, and the edits that most improve the
adversarial loss are selected. We show that adv-ICL results in significant
improvements over state-of-the-art prompt optimization techniques for both open
and closed-source models on 11 generation and classification tasks including
summarization, arithmetic reasoning, machine translation, data-to-text
generation, and the MMLU and big-bench hard benchmarks. In addition, because
our method uses pre-trained models and updates only prompts rather than model
parameters, it is computationally efficient, easy to extend to any LLM and
task, and effective in low-resource settings.
