  Computer-assisted surgery (CAS) aims to provide the surgeon with the right
type of assistance at the right moment. Such assistance systems are especially
relevant in laparoscopic surgery, where CAS can alleviate some of the drawbacks
that surgeons incur. For many assistance functions, e.g. displaying the
location of a tumor at the appropriate time or suggesting what instruments to
prepare next, analyzing the surgical workflow is a prerequisite. Since
laparoscopic interventions are performed via endoscope, the video signal is an
obvious sensor modality to rely on for workflow analysis.
  Image-based workflow analysis tasks in laparoscopy, such as phase
recognition, skill assessment, video indexing or automatic annotation, require
a temporal distinction between video frames. Generally computer vision based
methods that generalize from previously seen data are used. For training such
methods, large amounts of annotated data are necessary. Annotating surgical
data requires expert knowledge, therefore collecting a sufficient amount of
data is difficult, time-consuming and not always feasible.
  In this paper, we address this problem by presenting an unsupervised method
for training a convolutional neural network (CNN) to differentiate between
laparoscopic video frames on a temporal basis. We extract video frames at
regular intervals from 324 unlabeled laparoscopic interventions, resulting in a
dataset of approximately 2.2 million images. From this dataset, we extract
image pairs from the same video and train a CNN to determine their temporal
order. To solve this problem, the CNN has to extract features that are relevant
for comprehending laparoscopic workflow.
  Furthermore, we demonstrate that such a CNN can be adapted for surgical
workflow segmentation. We performed image-based workflow segmentation on a
publicly available dataset of 7 cholecystectomies and 9 colorectal
interventions.
