  Recent developments in large language models (LLMs) have unlocked new
opportunities for healthcare, from information synthesis to clinical decision
support. These new LLMs are not just capable of modeling language, but can also
act as intelligent "agents" that interact with stakeholders in open-ended
conversations and even influence clinical decision-making. Rather than relying
on benchmarks that measure a model's ability to process clinical data or answer
standardized test questions, LLM agents should be assessed for their
performance on real-world clinical tasks. These new evaluation frameworks,
which we call "Artificial-intelligence Structured Clinical Examinations"
("AI-SCI"), can draw from comparable technologies where machines operate with
varying degrees of self-governance, such as self-driving cars. High-fidelity
simulations may also be used to evaluate interactions between users and LLMs
within a clinical workflow, or to model the dynamic interactions of multiple
LLMs. Developing these robust, real-world clinical evaluations will be crucial
towards deploying LLM agents into healthcare.
