  The universality of deep neural networks across different modalities and
their generalization capabilities to unseen domains play an essential role in
medical image segmentation. The recent Segment Anything Model (SAM) has
demonstrated its potential in both settings. However, the huge computational
costs, demand for manual annotations as prompts and conflict-prone decoding
process of SAM degrade its generalizability and applicability in clinical
scenarios. To address these issues, we propose an efficient self-prompting SAM
for universal domain-generalized medical image segmentation, named ESP-MedSAM.
Specifically, we first devise the Multi-Modal Decoupled Knowledge Distillation
(MMDKD) strategy to construct a lightweight semi-parameter sharing image
encoder that produces discriminative visual features for diverse modalities.
Further, we introduce the Self-Patch Prompt Generator (SPPG) to automatically
generate high-quality dense prompt embeddings for guiding segmentation
decoding. Finally, we design the Query-Decoupled Modality Decoder (QDMD) that
leverages a one-to-one strategy to provide an independent decoding channel for
every modality. Extensive experiments indicate that ESP-MedSAM outperforms
state-of-the-arts in diverse medical imaging segmentation tasks, displaying
superior modality universality and generalization capabilities. Especially,
ESP-MedSAM uses only 4.5\% parameters compared to SAM-H. The source code is
available at https://github.com/xq141839/ESP-MedSAM.
