  We systematically evaluate a Deep Learning (DL) method in a 3D medical image
segmentation task. Our segmentation method is integrated into the radiosurgery
treatment process and directly impacts the clinical workflow. With our method,
we address the relative drawbacks of manual segmentation: high inter-rater
contouring variability and high time consumption of the contouring process. The
main extension over the existing evaluations is the careful and detailed
analysis that could be further generalized on other medical image segmentation
tasks. Firstly, we analyze the changes in the inter-rater detection agreement.
We show that the segmentation model reduces the ratio of detection
disagreements from 0.162 to 0.085 (p < 0.05). Secondly, we show that the model
improves the inter-rater contouring agreement from 0.845 to 0.871 surface Dice
Score (p < 0.05). Thirdly, we show that the model accelerates the delineation
process in between 1.6 and 2.0 times (p < 0.05). Finally, we design the setup
of the clinical experiment to either exclude or estimate the evaluation biases,
thus preserve the significance of the results. Besides the clinical evaluation,
we also summarize the intuitions and practical ideas for building an efficient
DL-based model for 3D medical image segmentation.
