  Computer-assisted minimally invasive surgery has great potential in
benefiting modern operating theatres. The video data streamed from the
endoscope provides rich information to support context-awareness for
next-generation intelligent surgical systems. To achieve accurate perception
and automatic manipulation during the procedure, learning based technique is a
promising way, which enables advanced image analysis and scene understanding in
recent years. However, learning such models highly relies on large-scale,
high-quality, and multi-task labelled data. This is currently a bottleneck for
the topic, as available public dataset is still extremely limited in the field
of CAI. In this paper, we present and release the first integrated dataset
(named AutoLaparo) with multiple image-based perception tasks to facilitate
learning-based automation in hysterectomy surgery. Our AutoLaparo dataset is
developed based on full-length videos of entire hysterectomy procedures.
Specifically, three different yet highly correlated tasks are formulated in the
dataset, including surgical workflow recognition, laparoscope motion
prediction, and instrument and key anatomy segmentation. In addition, we
provide experimental results with state-of-the-art models as reference
benchmarks for further model developments and evaluations on this dataset. The
dataset is available at https://autolaparo.github.io.
