  Surgical videos captured from microscopic or endoscopic imaging devices are
rich but complex sources of information, depicting different tools and
anatomical structures utilized during an extended amount of time. Despite
containing crucial workflow information and being commonly recorded in many
procedures, usage of surgical videos for automated surgical workflow
understanding is still limited.
  In this work, we exploit scene graphs as a more holistic, semantically
meaningful and human-readable way to represent surgical videos while encoding
all anatomical structures, tools, and their interactions. To properly evaluate
the impact of our solutions, we create a scene graph dataset from semantic
segmentations from the CaDIS and CATARACTS datasets. We demonstrate that scene
graphs can be leveraged through the use of graph convolutional networks (GCNs)
to tackle surgical downstream tasks such as surgical workflow recognition with
competitive performance. Moreover, we demonstrate the benefits of surgical
scene graphs regarding the explainability and robustness of model decisions,
which are crucial in the clinical setting.
