  Surgical workflow anticipation can give predictions on what steps to conduct
or what instruments to use next, which is an essential part of the
computer-assisted intervention system for surgery, e.g. workflow reasoning in
robotic surgery. However, current approaches are limited to their insufficient
expressive power for relationships between instruments. Hence, we propose a
graph representation learning framework to comprehensively represent instrument
motions in the surgical workflow anticipation problem. In our proposed graph
representation, we maps the bounding box information of instruments to the
graph nodes in the consecutive frames and build inter-frame/inter-instrument
graph edges to represent the trajectory and interaction of the instruments over
time. This design enhances the ability of our network on modeling both the
spatial and temporal patterns of surgical instruments and their interactions.
In addition, we design a multi-horizon learning strategy to balance the
understanding of various horizons indifferent anticipation tasks, which
significantly improves the model performance in anticipation with various
horizons. Experiments on the Cholec80 dataset demonstrate the performance of
our proposed method can exceed the state-of-the-art method based on richer
backbones, especially in instrument anticipation (1.27 v.s. 1.48 for inMAE;
1.48 v.s. 2.68 for eMAE). To the best of our knowledge, we are the first to
introduce a spatial-temporal graph representation into surgical workflow
anticipation.
