  The surgical intervention is crucial to patient healthcare, and many studies
have developed advanced algorithms to provide understanding and decision-making
assistance for surgeons. Despite great progress, these algorithms are developed
for a single specific task and scenario, and in practice require the manual
combination of different functions, thus limiting the applicability. Thus, an
intelligent and versatile surgical assistant is expected to accurately
understand the surgeon's intentions and accordingly conduct the specific tasks
to support the surgical process. In this work, by leveraging advanced
multimodal large language models (MLLMs), we propose a Versatile Surgery
Assistant (VS-Assistant) that can accurately understand the surgeon's intention
and complete a series of surgical understanding tasks, e.g., surgical scene
analysis, surgical instrument detection, and segmentation on demand.
Specifically, to achieve superior surgical multimodal understanding, we devise
a mixture of projectors (MOP) module to align the surgical MLLM in VS-Assistant
to balance the natural and surgical knowledge. Moreover, we devise a surgical
Function-Calling Tuning strategy to enable the VS-Assistant to understand
surgical intentions, and thus make a series of surgical function calls on
demand to meet the needs of the surgeons. Extensive experiments on neurosurgery
data confirm that our VS-Assistant can understand the surgeon's intention more
accurately than the existing MLLM, resulting in overwhelming performance in
textual analysis and visual tasks. Source code and models will be made public.
