  In-context decision-making is an important capability of artificial general
intelligence, which Large Language Models (LLMs) have effectively demonstrated
in various scenarios. However, LLMs often face challenges when dealing with
numerical contexts, and limited attention has been paid to evaluating their
performance through preference feedback generated by the environment. This
paper investigates the performance of LLMs as decision-makers in the context of
Dueling Bandits (DB). We first evaluate the performance of LLMs by comparing
GPT-3.5-Turbo, GPT-4, and GPT-4-Turbo against established DB algorithms. Our
results reveal that LLMs, particularly GPT-4 Turbo, quickly identify the
Condorcet winner, thus outperforming existing state-of-the-art algorithms in
terms of weak regret. Nevertheless, LLMs struggle to converge even when
explicitly prompted to do so, and are sensitive to prompt variations. To
overcome these issues, we introduce an LLM-augmented algorithm, IF-Enhanced
LLM, which takes advantage of both in-context decision-making capabilities of
LLMs and theoretical guarantees inherited from classic DB algorithms. The
design of such an algorithm sheds light on how to enhance trustworthiness for
LLMs used in decision-making tasks where performance robustness matters. We
show that IF-Enhanced LLM has theoretical guarantees on both weak and strong
regret. Our experimental results validate that IF-Enhanced LLM is robust even
with noisy and adversarial prompts.
