  Despite the remarkable success of deep learning in medical imaging analysis,
medical image segmentation remains challenging due to the scarcity of
high-quality labeled images for supervision. Further, the significant domain
gap between natural and medical images in general and ultrasound images in
particular hinders fine-tuning models trained on natural images to the task at
hand. In this work, we address the performance degradation of segmentation
models in low-data regimes and propose a prompt-less segmentation method
harnessing the ability of segmentation foundation models to segment abstract
shapes. We do that via our novel prompt point generation algorithm which uses
coarse semantic segmentation masks as input and a zero-shot prompt-able
foundation model as an optimization target. We demonstrate our method on a
segmentation findings task (pathologic anomalies) in ultrasound images. Our
method's advantages are brought to light in varying degrees of low-data regime
experiments on a small-scale musculoskeletal ultrasound images dataset,
yielding a larger performance gain as the training set size decreases.
