  Machine learning-based segmentation in medical imaging is widely used in
clinical applications from diagnostics to radiotherapy treatment planning.
Segmented medical images with ground truth are useful for investigating the
properties of different segmentation performance metrics to inform metric
selection. Regular geometrical shapes are often used to synthesize segmentation
errors and illustrate properties of performance metrics, but they lack the
complexity of anatomical variations in real images. In this study, we present a
tool to emulate segmentations by adjusting the reference (truth) masks of
anatomical objects extracted from real medical images. Our tool is designed to
modify the defined truth contours and emulate different types of segmentation
errors with a set of user-configurable parameters. We defined the ground truth
objects from 230 patient images in the Glioma Image Segmentation for
Radiotherapy (GLIS-RT) database. For each object, we used our segmentation
synthesis tool to synthesize 10 versions of segmentation (i.e., 10 simulated
segmentors or algorithms), where each version has a pre-defined combination of
segmentation errors. We then applied 20 performance metrics to evaluate all
synthetic segmentations. We demonstrated the properties of these metrics,
including their ability to capture specific types of segmentation errors. By
analyzing the intrinsic properties of these metrics and categorizing the
segmentation errors, we are working toward the goal of developing a
decision-tree tool for assisting in the selection of segmentation performance
metrics.
