  Real-time algorithms for automatically recognizing surgical phases are needed
to develop systems that can provide assistance to surgeons, enable better
management of operating room (OR) resources and consequently improve safety
within the OR. State-of-the-art surgical phase recognition algorithms using
laparoscopic videos are based on fully supervised training. This limits their
potential for widespread application, since creation of manual annotations is
an expensive process considering the numerous types of existing surgeries and
the vast amount of laparoscopic videos available. In this work, we propose a
new self-supervised pre-training approach based on the prediction of remaining
surgery duration (RSD) from laparoscopic videos. The RSD prediction task is
used to pre-train a convolutional neural network (CNN) and long short-term
memory (LSTM) network in an end-to-end manner. Our proposed approach utilizes
all available data and reduces the reliance on annotated data, thereby
facilitating the scaling up of surgical phase recognition algorithms to
different kinds of surgeries. Additionally, we present EndoN2N, an end-to-end
trained CNN-LSTM model for surgical phase recognition and evaluate the
performance of our approach on a dataset of 120 Cholecystectomy laparoscopic
videos (Cholec120). This work also presents the first systematic study of
self-supervised pre-training approaches to understand the amount of annotations
required for surgical phase recognition. Interestingly, the proposed RSD
pre-training approach leads to performance improvement even when all the
training data is manually annotated and outperforms the single pre-training
approach for surgical phase recognition presently published in the literature.
It is also observed that end-to-end training of CNN-LSTM networks boosts
surgical phase recognition performance.
