  Delineating lesions and anatomical structure is important for image-guided
interventions. Point-supervised medical image segmentation (PSS) has great
potential to alleviate costly expert delineation labeling. However, due to the
lack of precise size and boundary guidance, the effectiveness of PSS often
falls short of expectations. Although recent vision foundational models, such
as the medical segment anything model (MedSAM), have made significant
advancements in bounding-box-prompted segmentation, it is not straightforward
to utilize point annotation, and is prone to semantic ambiguity. In this
preliminary study, we introduce an iterative framework to facilitate
semantic-aware point-supervised MedSAM. Specifically, the semantic box-prompt
generator (SBPG) module has the capacity to convert the point input into
potential pseudo bounding box suggestions, which are explicitly refined by the
prototype-based semantic similarity. This is then succeeded by a prompt-guided
spatial refinement (PGSR) module that harnesses the exceptional
generalizability of MedSAM to infer the segmentation mask, which also updates
the box proposal seed in SBPG. Performance can be progressively improved with
adequate iterations. We conducted an evaluation on BraTS2018 for the
segmentation of whole brain tumors and demonstrated its superior performance
compared to traditional PSS methods and on par with box-supervised methods.
