  Semantic segmentation is a complex task that relies heavily on large amounts
of annotated image data. However, annotating such data can be time-consuming
and resource-intensive, especially in the medical domain. Active Learning (AL)
is a popular approach that can help to reduce this burden by iteratively
selecting images for annotation to improve the model performance. In the case
of video data, it is important to consider the model uncertainty and the
temporal nature of the sequences when selecting images for annotation. This
work proposes a novel AL strategy for surgery video segmentation, COWAL,
COrrelation-aWare Active Learning. Our approach involves projecting images into
a latent space that has been fine-tuned using contrastive learning and then
selecting a fixed number of representative images from local clusters of video
frames. We demonstrate the effectiveness of this approach on two video datasets
of surgical instruments and three real-world video datasets. The datasets and
code will be made publicly available upon receiving necessary approvals.
