  We explore the potential of Large Language Models (LLMs) to assist and
potentially correct physicians in medical decision-making tasks. We evaluate
several LLMs, including Meditron, Llama2, and Mistral, to analyze the ability
of these models to interact effectively with physicians across different
scenarios. We consider questions from PubMedQA and several tasks, ranging from
binary (yes/no) responses to long answer generation, where the answer of the
model is produced after an interaction with a physician. Our findings suggest
that prompt design significantly influences the downstream accuracy of LLMs and
that LLMs can provide valuable feedback to physicians, challenging incorrect
diagnoses and contributing to more accurate decision-making. For example, when
the physician is accurate 38% of the time, Mistral can produce the correct
answer, improving accuracy up to 74% depending on the prompt being used, while
Llama2 and Meditron models exhibit greater sensitivity to prompt choice. Our
analysis also uncovers the challenges of ensuring that LLM-generated
suggestions are pertinent and useful, emphasizing the need for further research
in this area.
