  In this paper, we examine the recent Segment Anything Model (SAM) on medical
images, and report both quantitative and qualitative zero-shot segmentation
results on nine medical image segmentation benchmarks, covering various imaging
modalities, such as optical coherence tomography (OCT), magnetic resonance
imaging (MRI), and computed tomography (CT), as well as different applications
including dermatology, ophthalmology, and radiology. Those benchmarks are
representative and commonly used in model development. Our experimental results
indicate that while SAM presents remarkable segmentation performance on images
from the general domain, its zero-shot segmentation ability remains restricted
for out-of-distribution images, e.g., medical images. In addition, SAM exhibits
inconsistent zero-shot segmentation performance across different unseen medical
domains. For certain structured targets, e.g., blood vessels, the zero-shot
segmentation of SAM completely failed. In contrast, a simple fine-tuning of it
with a small amount of data could lead to remarkable improvement of the
segmentation quality, showing the great potential and feasibility of using
fine-tuned SAM to achieve accurate medical image segmentation for a precision
diagnostics. Our study indicates the versatility of generalist vision
foundation models on medical imaging, and their great potential to achieve
desired performance through fine-turning and eventually address the challenges
associated with accessing large and diverse medical datasets in support of
clinical diagnostics.
