  The Segment Anything Model (SAM) exhibits promise in generic object
segmentation and offers potential for various applications. Existing methods
have applied SAM to surgical instrument segmentation (SIS) by tuning SAM-based
frameworks with surgical data. However, they fall short in two crucial aspects:
(1) Straightforward model tuning with instrument masks treats each instrument
as a single entity, neglecting their complex structures and fine-grained
details; and (2) Instrument category-based prompts are not flexible and
informative enough to describe instrument structures. To address these
problems, in this paper, we investigate text promptable SIS and propose
SurgicalPart-SAM (SP-SAM), a novel SAM efficient-tuning approach that
explicitly integrates instrument structure knowledge with SAM's generic
knowledge, guided by expert knowledge on instrument part compositions.
Specifically, we achieve this by proposing (1) Collaborative Prompts that
describe instrument structures via collaborating category-level and part-level
texts; (2) Cross-Modal Prompt Encoder that encodes text prompts jointly with
visual embeddings into discriminative part-level representations; and (3)
Part-to-Whole Adaptive Fusion and Hierarchical Decoding that adaptively fuse
the part-level representations into a whole for accurate instrument
segmentation in surgical scenarios. Built upon them, SP-SAM acquires a better
capability to comprehend surgical instruments in terms of both overall
structure and part-level details. Extensive experiments on both the EndoVis2018
and EndoVis2017 datasets demonstrate SP-SAM's state-of-the-art performance with
minimal tunable parameters. The code will be available at
https://github.com/wenxi-yue/SurgicalPart-SAM.
