  The Segment Anything Model (SAM) is a powerful foundation model that has
revolutionised image segmentation. To apply SAM to surgical instrument
segmentation, a common approach is to locate precise points or boxes of
instruments and then use them as prompts for SAM in a zero-shot manner.
However, we observe two problems with this naive pipeline: (1) the domain gap
between natural objects and surgical instruments leads to inferior
generalisation of SAM; and (2) SAM relies on precise point or box locations for
accurate segmentation, requiring either extensive manual guidance or a
well-performing specialist detector for prompt preparation, which leads to a
complex multi-stage pipeline. To address these problems, we introduce
SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to
effectively integrate surgical-specific information with SAM's pre-trained
knowledge for improved generalisation. Specifically, we propose a lightweight
prototype-based class prompt encoder for tuning, which directly generates
prompt embeddings from class prototypes and eliminates the use of explicit
prompts for improved robustness and a simpler pipeline. In addition, to address
the low inter-class variance among surgical instrument categories, we propose
contrastive prototype learning, further enhancing the discrimination of the
class prototypes for more accurate class prompting. The results of extensive
experiments on both EndoVis2018 and EndoVis2017 datasets demonstrate that
SurgicalSAM achieves state-of-the-art performance while only requiring a small
number of tunable parameters. The source code is available at
https://github.com/wenxi-yue/SurgicalSAM.
