  The "pre-training then fine-tuning (FT)" paradigm is widely adopted to boost
the model performance of deep learning-based methods for medical volumetric
segmentation. However, conventional full FT incurs high computational and
memory costs. Thus, it is of increasing importance to fine-tune pre-trained
models for medical volumetric segmentation tasks in a both effective and
parameter-efficient manner. In this paper, we introduce a new framework named
Med-Tuning to realize parameter-efficient tuning (PET) for medical volumetric
segmentation task and an efficient plug-and-play module named Med-Adapter for
task-specific feature extraction. With a small number of tuned parameters, our
framework enhances the 2D baselines's precision on segmentation tasks, which
are pre-trained on natural images. Extensive experiments on three benchmark
datasets (CT and MRI modalities) show that our method achieves better results
than previous PET methods on volumetric segmentation tasks. Compared to full
FT, Med-Tuning reduces the fine-tuned model parameters by up to 4x, with even
better segmentation performance. Our project webpage is at
\url{https://rubics-xuan.github.io/Med-Tuning/}.
