  Despite significant advancements in robotic systems and surgical data
science, ensuring safe and optimal execution in robot-assisted minimally
invasive surgery (RMIS) remains a complex challenge. Current surgical error
detection methods involve two parts: identifying surgical gestures and then
detecting errors within each gesture clip. These methods seldom consider the
rich contextual and semantic information inherent in surgical videos, limiting
their performance due to reliance on accurate gesture identification. Motivated
by the chain-of-thought prompting in natural language processing, this letter
presents a novel and real-time end-to-end error detection framework,
Chain-of-Thought (COG) prompting, leveraging contextual information from
surgical videos. This encompasses two reasoning modules designed to mimic the
decision-making processes of expert surgeons. Concretely, we first design a
Gestural-Visual Reasoning module, which utilizes transformer and attention
architectures for gesture prompting, while the second, a Multi-Scale Temporal
Reasoning module, employs a multi-stage temporal convolutional network with
both slow and fast paths for temporal information extraction. We extensively
validate our method on the public benchmark RMIS dataset JIGSAWS. Our method
encapsulates the reasoning processes inherent to surgical activities enabling
it to outperform the state-of-the-art by 4.6% in F1 score, 4.6% in Accuracy,
and 5.9% in Jaccard index while processing each frame in 6.69 milliseconds on
average, demonstrating the great potential of our approach in enhancing the
safety and efficacy of RMIS procedures and surgical education. The code will be
available.
