  High resolution computed tomography (HRCT) is the most important imaging
modality for interstitial lung diseases, where the radiologists are interested
in identifying certain patterns, and their volumetric and regional
distribution. The use of machine learning can assist the radiologists with both
these tasks by performing semantic segmentation. In this paper, we propose an
interactive annotation-tool for semantic segmentation that assists the
radiologist in labeling CT scans. The annotation tool is evaluated by six
radiologists and radiology residents classifying healthy lung and reticular
pattern i HRCT images. The usability of the system is evaluated with a System
Usability Score (SUS) and interaction information from the readers that used
the tool for annotating the CT volumes. It was discovered that the experienced
usability and how the users interactied with the system differed between the
users. A higher SUS-score was given by users that prioritized learning speed
over model accuracy and spent less time with manual labeling and instead
utilized the suggestions provided by the GUI. An analysis of the annotation
variations between the readers show substantial agreement (Cohen's kappa=0.69)
for classification of healthy and affected lung parenchyma in pulmonary
fibrosis. The inter-reader variation is a challenge for the definition of
ground truth.
