  It has been shown that traditional deep learning methods for electronic
microscopy segmentation usually suffer from low transferability when samples
and annotations are limited, while large-scale vision foundation models are
more robust when transferring between different domains but facing sub-optimal
improvement under fine-tuning. In this work, we present a new few-shot domain
adaptation framework SAMDA, which combines the Segment Anything Model(SAM) with
nnUNet in the embedding space to achieve high transferability and accuracy.
Specifically, we choose the Unet-based network as the "expert" component to
learn segmentation features efficiently and design a SAM-based adaptation
module as the "generic" component for domain transfer. By amalgamating the
"generic" and "expert" components, we mitigate the modality imbalance in the
complex pre-training knowledge inherent to large-scale Vision Foundation models
and the challenge of transferability inherent to traditional neural networks.
The effectiveness of our model is evaluated on two electron microscopic image
datasets with different modalities for mitochondria segmentation, which
improves the dice coefficient on the target domain by 6.7%. Also, the SAM-based
adaptor performs significantly better with only a single annotated image than
the 10-shot domain adaptation on nnUNet. We further verify our model on four
MRI datasets from different sources to prove its generalization ability.
