  Segment Anything Model (SAM) is a foundation model for semantic segmentation
and shows excellent generalization capability with the prompts. In this
empirical study, we investigate the robustness and zero-shot generalizability
of the SAM in the domain of robotic surgery in various settings of (i) prompted
vs. unprompted; (ii) bounding box vs. points-based prompt; (iii) generalization
under corruptions and perturbations with five severity levels; and (iv)
state-of-the-art supervised model vs. SAM. We conduct all the observations with
two well-known robotic instrument segmentation datasets of MICCAI EndoVis 2017
and 2018 challenges. Our extensive evaluation results reveal that although SAM
shows remarkable zero-shot generalization ability with bounding box prompts, it
struggles to segment the whole instrument with point-based prompts and
unprompted settings. Furthermore, our qualitative figures demonstrate that the
model either failed to predict the parts of the instrument mask (e.g., jaws,
wrist) or predicted parts of the instrument as different classes in the
scenario of overlapping instruments within the same bounding box or with the
point-based prompt. In fact, it is unable to identify instruments in some
complex surgical scenarios of blood, reflection, blur, and shade. Additionally,
SAM is insufficiently robust to maintain high performance when subjected to
various forms of data corruption. Therefore, we can argue that SAM is not ready
for downstream surgical tasks without further domain-specific fine-tuning.
