  Radiotherapy treatment planning is a time-consuming and potentially
subjective process that requires the iterative adjustment of model parameters
to balance multiple conflicting objectives. Recent advancements in large
foundation models offer promising avenues for addressing the challenges in
planning and clinical decision-making. This study introduces GPT-RadPlan, a
fully automated treatment planning framework that harnesses prior radiation
oncology knowledge encoded in multi-modal large language models, such as
GPT-4Vision (GPT-4V) from OpenAI. GPT-RadPlan is made aware of planning
protocols as context and acts as an expert human planner, capable of guiding a
treatment planning process. Via in-context learning, we incorporate clinical
protocols for various disease sites as prompts to enable GPT-4V to acquire
treatment planning domain knowledge. The resulting GPT-RadPlan agent is
integrated into our in-house inverse treatment planning system through an API.
The efficacy of the automated planning system is showcased using multiple
prostate and head & neck cancer cases, where we compared GPT-RadPlan results to
clinical plans. In all cases, GPT-RadPlan either outperformed or matched the
clinical plans, demonstrating superior target coverage and organ-at-risk
sparing. Consistently satisfying the dosimetric objectives in the clinical
protocol, GPT-RadPlan represents the first multimodal large language model
agent that mimics the behaviors of human planners in radiation oncology
clinics, achieving remarkable results in automating the treatment planning
process without the need for additional training.
