  This study performs comprehensive evaluation of four neural network
architectures (UNet, SegResNet, DynUNet, and SwinUNETR) for lymphoma lesion
segmentation from PET/CT images. These networks were trained, validated, and
tested on a diverse, multi-institutional dataset of 611 cases. Internal testing
(88 cases; total metabolic tumor volume (TMTV) range [0.52, 2300] ml) showed
SegResNet as the top performer with a median Dice similarity coefficient (DSC)
of 0.76 and median false positive volume (FPV) of 4.55 ml; all networks had a
median false negative volume (FNV) of 0 ml. On the unseen external test set
(145 cases with TMTV range: [0.10, 2480] ml), SegResNet achieved the best
median DSC of 0.68 and FPV of 21.46 ml, while UNet had the best FNV of 0.41 ml.
We assessed reproducibility of six lesion measures, calculated their prediction
errors, and examined DSC performance in relation to these lesion measures,
offering insights into segmentation accuracy and clinical relevance.
Additionally, we introduced three lesion detection criteria, addressing the
clinical need for identifying lesions, counting them, and segmenting based on
metabolic characteristics. We also performed expert intra-observer variability
analysis revealing the challenges in segmenting ``easy'' vs. ``hard'' cases, to
assist in the development of more resilient segmentation algorithms. Finally,
we performed inter-observer agreement assessment underscoring the importance of
a standardized ground truth segmentation protocol involving multiple expert
annotators. Code is available at:
https://github.com/microsoft/lymphoma-segmentation-dnn
