  This study examines the effect of prompt engineering on the performance of
Large Language Models (LLMs) in clinical note generation. We introduce an
Automatic Prompt Optimization (APO) framework to refine initial prompts and
compare the outputs of medical experts, non-medical experts, and APO-enhanced
GPT3.5 and GPT4. Results highlight GPT4 APO's superior performance in
standardizing prompt quality across clinical note sections. A human-in-the-loop
approach shows that experts maintain content quality post-APO, with a
preference for their own modifications, suggesting the value of expert
customization. We recommend a two-phase optimization process, leveraging
APO-GPT4 for consistency and expert input for personalization.
