  The Segment Anything Model (SAM) has achieved remarkable successes in the
realm of natural image segmentation, but its deployment in the medical imaging
sphere has encountered challenges. Specifically, the model struggles with
medical images that feature low contrast, faint boundaries, intricate
morphologies, and small-sized objects. To address these challenges and enhance
SAM's performance in the medical domain, we introduce a comprehensive
modification. Firstly, we incorporate a frozen Convolutional Neural Network
(CNN) branch as an image encoder, which synergizes with SAM's original Vision
Transformer (ViT) encoder through a novel variational attention fusion module.
This integration bolsters the model's capability to capture local spatial
information, which is often paramount in medical imagery. Moreover, to further
optimize SAM for medical imaging, we introduce feature and position adapters
within the ViT branch, refining the encoder's representations. We see that
compared to current prompting strategies to fine-tune SAM for ultrasound
medical segmentation, the use of text descriptions that serve as text prompts
for SAM helps significantly improve the performance. Leveraging ChatGPT's
natural language understanding capabilities, we generate prompts that offer
contextual information and guidance to SAM, enabling it to better understand
the nuances of ultrasound medical images and improve its segmentation accuracy.
Our method, in its entirety, represents a significant stride towards making
universal image segmentation models more adaptable and efficient in the medical
domain.
