  Surgical tool segmentation in endoscopic images is an important problem: it
is a crucial step towards full instrument pose estimation and it is used for
integration of pre- and intra-operative images into the endoscopic view. While
many recent approaches based on convolutional neural networks have shown great
results, a key barrier to progress lies in the acquisition of a large number of
manually-annotated images which is necessary for an algorithm to generalize and
work well in diverse surgical scenarios. Unlike the surgical image data itself,
annotations are difficult to acquire and may be of variable quality. On the
other hand, synthetic annotations can be automatically generated by using
forward kinematic model of the robot and CAD models of tools by projecting them
onto an image plane. Unfortunately, this model is very inaccurate and cannot be
used for supervised learning of image segmentation models. Since generated
annotations will not directly correspond to endoscopic images due to errors, we
formulate the problem as an unpaired image-to-image translation where the goal
is to learn the mapping between an input endoscopic image and a corresponding
annotation using an adversarial model. Our approach allows to train image
segmentation models without the need to acquire expensive annotations and can
potentially exploit large unlabeled endoscopic image collection outside the
annotated distributions of image/annotation data. We test our proposed method
on Endovis 2017 challenge dataset and show that it is competitive with
supervised segmentation methods.
