  We evaluated the temporal performance of a deep learning (DL) based
artificial intelligence (AI) model for auto segmentation in prostate
radiotherapy, seeking to correlate its efficacy with changes in clinical
landscapes. Our study involved 1328 prostate cancer patients who underwent
definitive radiotherapy from January 2006 to August 2022 at the University of
Texas Southwestern Medical Center. We trained a UNet based segmentation model
on data from 2006 to 2011 and tested it on data from 2012 to 2022 to simulate
real world clinical deployment. We measured the model performance using the
Dice similarity coefficient (DSC), visualized the trends in contour quality
using exponentially weighted moving average (EMA) curves. Additionally, we
performed Wilcoxon Rank Sum Test to analyze the differences in DSC
distributions across distinct periods, and multiple linear regression to
investigate the impact of various clinical factors. The model exhibited peak
performance in the initial phase (from 2012 to 2014) for segmenting the
prostate, rectum, and bladder. However, we observed a notable decline in
performance for the prostate and rectum after 2015, while bladder contour
quality remained stable. Key factors that impacted the prostate contour quality
included physician contouring styles, the use of various hydrogel spacer, CT
scan slice thickness, MRI-guided contouring, and using intravenous (IV)
contrast. Rectum contour quality was influenced by factors such as slice
thickness, physician contouring styles, and the use of various hydrogel
spacers. The bladder contour quality was primarily affected by using IV
contrast. This study highlights the challenges in maintaining AI model
performance consistency in a dynamic clinical setting. It underscores the need
for continuous monitoring and updating of AI models to ensure their ongoing
effectiveness and relevance in patient care.
