  Large Language Models (LLMs) have gained widespread popularity due to their
ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple
natural language prompt. Part of the appeal for LLMs is their approachability
to the general public, including individuals with no prior technical experience
in NLP techniques. However, natural language prompts can vary significantly in
terms of their linguistic structure, context, and other semantics. Modifying
one or more of these aspects can result in significant differences in task
performance. Non-expert users may find it challenging to identify the changes
needed to improve a prompt, especially when they lack domain-specific knowledge
and lack appropriate feedback. To address this challenge, we present PromptAid,
a visual analytics system designed to interactively create, refine, and test
prompts through exploration, perturbation, testing, and iteration. PromptAid
uses multiple, coordinated visualizations which allow users to improve prompts
by using the three strategies: keyword perturbations, paraphrasing
perturbations, and obtaining the best set of in-context few-shot examples.
PromptAid was designed through an iterative prototyping process involving NLP
experts and was evaluated through quantitative and qualitative assessments for
LLMs. Our findings indicate that PromptAid helps users to iterate over prompt
template alterations with less cognitive overhead, generate diverse prompts
with help of recommendations, and analyze the performance of the generated
prompts while surpassing existing state-of-the-art prompting interfaces in
performance.
