  Weakly supervised surgical instrument segmentation with only instrument
presence labels has been rarely explored in surgical domain. To mitigate the
highly under-constrained challenges, we extend a two-stage weakly supervised
segmentation paradigm with temporal attributes from two perspectives. From a
temporal equivariance perspective, we propose a prototype-based temporal
equivariance regulation loss to enhance pixel-wise consistency between adjacent
features. From a semantic continuity perspective, we propose a class-aware
temporal semantic continuity loss to constrain the semantic consistency between
a global view of target frame and local non-discriminative regions of adjacent
reference frame. To the best of our knowledge, WeakSurg is the first
instrument-presence-only weakly supervised segmentation architecture to take
temporal information into account for surgical scenarios. Extensive experiments
are validated on Cholec80, an open benchmark for phase and instrument
recognition. We annotate instance-wise instrument labels with fixed time-steps
which are double checked by a clinician with 3-years experience. Our results
show that WeakSurg compares favorably with state-of-the-art methods not only on
semantic segmentation metrics but also on instance segmentation metrics.
