  Online surgical phase recognition plays a significant role towards building
contextual tools that could quantify performance and oversee the execution of
surgical workflows. Current approaches are limited since they train spatial
feature extractors using frame-level supervision that could lead to incorrect
predictions due to similar frames appearing at different phases, and poorly
fuse local and global features due to computational constraints which can
affect the analysis of long videos commonly encountered in surgical
interventions. In this paper, we present a two-stage method, called Long Video
Transformer (LoViT) for fusing short- and long-term temporal information that
combines a temporally-rich spatial feature extractor and a multi-scale temporal
aggregator consisting of two cascaded L-Trans modules based on self-attention,
followed by a G-Informer module based on ProbSparse self-attention for
processing global temporal information. The multi-scale temporal head then
combines local and global features and classifies surgical phases using phase
transition-aware supervision. Our approach outperforms state-of-the-art methods
on the Cholec80 and AutoLaparo datasets consistently. Compared to Trans-SVNet,
LoViT achieves a 2.4 pp (percentage point) improvement in video-level accuracy
on Cholec80 and a 3.1 pp improvement on AutoLaparo. Moreover, it achieves a 5.3
pp improvement in phase-level Jaccard on AutoLaparo and a 1.55 pp improvement
on Cholec80. Our results demonstrate the effectiveness of our approach in
achieving state-of-the-art performance of surgical phase recognition on two
datasets of different surgical procedures and temporal sequencing
characteristics whilst introducing mechanisms that cope with long videos.
