  Annotation and labeling of images are some of the biggest challenges in
applying deep learning to medical data. Current processes are time and
cost-intensive and, therefore, a limiting factor for the wide adoption of the
technology. Additionally validating that measured performance improvements are
significant is important to select the best model. In this paper, we
demonstrate a method for creating segmentations, a necessary part of a data
cleaning for ultrasound imaging machine learning pipelines. We propose a
four-step method to leverage automatically generated training data and fast
human visual checks to improve model accuracy while keeping the time/effort and
cost low. We also showcase running experiments multiple times to allow the
usage of statistical analysis. Poor quality automated ground truth data and
quick visual inspections efficiently train an initial base model, which is
refined using a small set of more expensive human-generated ground truth data.
The method is demonstrated on a cardiac ultrasound segmentation task, removing
background data, including static PHI. Significance is shown by running the
experiments multiple times and using the student's t-test on the performance
distributions. The initial segmentation accuracy of a simple thresholding
algorithm of 92% was improved to 98%. The performance of models trained on
complicated algorithms can be matched or beaten by pre-training with the poorer
performing algorithms and a small quantity of high-quality data. The
introduction of statistic significance analysis for deep learning models helps
to validate the performance improvements measured. The method offers a
cost-effective and fast approach to achieving high-accuracy models while
minimizing the cost and effort of acquiring high-quality training data.
