  To address prevalent issues in medical imaging, such as data acquisition
challenges and label availability, transfer learning from natural to medical
image domains serves as a viable strategy to produce reliable segmentation
results. However, several existing barriers between domains need to be broken
down, including addressing contrast discrepancies, managing anatomical
variability, and adapting 2D pretrained models for 3D segmentation tasks. In
this paper, we propose ProMISe,a prompt-driven 3D medical image segmentation
model using only a single point prompt to leverage knowledge from a pretrained
2D image foundation model. In particular, we use the pretrained vision
transformer from the Segment Anything Model (SAM) and integrate lightweight
adapters to extract depth-related (3D) spatial context without updating the
pretrained weights. For robust results, a hybrid network with complementary
encoders is designed, and a boundary-aware loss is proposed to achieve precise
boundaries. We evaluate our model on two public datasets for colon and pancreas
tumor segmentations, respectively. Compared to the state-of-the-art
segmentation methods with and without prompt engineering, our proposed method
achieves superior performance. The code is publicly available at
https://github.com/MedICL-VU/ProMISe.
