  Large language models (LLMs) have revolutionized AI across diverse domains,
showcasing remarkable capabilities. Central to their success is the concept of
prompting, which guides model output generation. However, manual prompt
engineering is labor-intensive and domain-specific, necessitating automated
solutions. This paper introduces PromptWizard, a novel framework leveraging
LLMs to iteratively synthesize and refine prompts tailored to specific tasks.
Unlike existing approaches, PromptWizard optimizes both prompt instructions and
in-context examples, maximizing model performance. The framework iteratively
refines prompts by mutating instructions and incorporating negative examples to
deepen understanding and ensure diversity. It further enhances both
instructions and examples with the aid of a critic, synthesizing new
instructions and examples enriched with detailed reasoning steps for optimal
performance. PromptWizard offers several key features and capabilities,
including computational efficiency compared to state-of-the-art approaches,
adaptability to scenarios with varying amounts of training data, and
effectiveness with smaller LLMs. Rigorous evaluation across 35 tasks on 8
datasets demonstrates PromptWizard's superiority over existing prompt
strategies, showcasing its efficacy and scalability in prompt optimization.
