  The accurate reconstruction of surgical scenes from surgical videos is
critical for various applications, including intraoperative navigation and
image-guided robotic surgery automation. However, previous approaches, mainly
relying on depth estimation, have limited effectiveness in reconstructing
surgical scenes with moving surgical tools. To address this limitation and
provide accurate 3D position prediction for surgical tools in all frames, we
propose a novel approach called SAMSNeRF that combines Segment Anything Model
(SAM) and Neural Radiance Field (NeRF) techniques. Our approach generates
accurate segmentation masks of surgical tools using SAM, which guides the
refinement of the dynamic surgical scene reconstruction by NeRF. Our
experimental results on public endoscopy surgical videos demonstrate that our
approach successfully reconstructs high-fidelity dynamic surgical scenes and
accurately reflects the spatial information of surgical tools. Our proposed
approach can significantly enhance surgical navigation and automation by
providing surgeons with accurate 3D position information of surgical tools
during surgery.The source code will be released soon.
