  Malignant pleural mesothelioma (MPM) is the most common form of mesothelioma.
To assess response to treatment, tumor measurements are acquired and evaluated
based on a patient's longitudinal computed tomography (CT) scans. Tumor volume,
however, is the more accurate metric for assessing tumor burden and response.
Automated segmentation methods using deep learning can be employed to acquire
volume, which otherwise is a tedious task performed manually. The deep
learning-based tumor volume and contours can then be compared with a standard
reference to assess the robustness of the automated segmentations. The purpose
of this study was to evaluate the impact of probability map threshold on MPM
tumor delineations generated using a convolutional neural network (CNN).
Eighty-eight CT scans from 21 MPM patients were segmented by a VGG16/U-Net CNN.
A radiologist modified the contours generated at a 0.5 probability threshold.
Percent difference of tumor volume and overlap using the Dice Similarity
Coefficient (DSC) were compared between the standard reference provided by the
radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN
annotations consistently yielded smaller tumor volumes than radiologist
contours. Reducing the probability threshold from 0.5 to 0.1 decreased the
absolute percent volume difference, on average, from 43.96% to 24.18%. Median
and mean DSC ranged from 0.58 to 0.60, with a peak at a threshold of 0.5; no
distinct threshold was found for percent volume difference. No single output
threshold in the CNN probability maps was optimal for both tumor volume and
DSC. This work underscores the need to assess tumor volume and spatial overlap
when evaluating CNN performance. While automated segmentations may yield
comparable tumor volumes to that of the reference standard, the spatial region
delineated by the CNN at a specific threshold is equally important.
