  The Segment Anything Model (SAM) is a recently proposed prompt-based
segmentation model in a generic zero-shot segmentation approach. With the
zero-shot segmentation capacity, SAM achieved impressive flexibility and
precision on various segmentation tasks. However, the current pipeline requires
manual prompts during the inference stage, which is still resource intensive
for biomedical image segmentation. In this paper, instead of using prompts
during the inference stage, we introduce a pipeline that utilizes the SAM,
called all-in-SAM, through the entire AI development workflow (from annotation
generation to model finetuning) without requiring manual prompts during the
inference stage. Specifically, SAM is first employed to generate pixel-level
annotations from weak prompts (e.g., points, bounding box). Then, the
pixel-level annotations are used to finetune the SAM segmentation model rather
than training from scratch. Our experimental results reveal two key findings:
1) the proposed pipeline surpasses the state-of-the-art (SOTA) methods in a
nuclei segmentation task on the public Monuseg dataset, and 2) the utilization
of weak and few annotations for SAM finetuning achieves competitive performance
compared to using strong pixel-wise annotated data.
