  Small-sized unmanned surface vehicles (USV) are coastal water devices with a
broad range of applications such as environmental control and surveillance. A
crucial capability for autonomous operation is obstacle detection for timely
reaction and collision avoidance, which has been recently explored in the
context of camera-based visual scene interpretation. Owing to curated datasets,
substantial advances in scene interpretation have been made in a related field
of unmanned ground vehicles. However, the current maritime datasets do not
adequately capture the complexity of real-world USV scenes and the evaluation
protocols are not standardised, which makes cross-paper comparison of different
methods difficult and hinders the progress. To address these issues, we
introduce a new obstacle detection benchmark MODS, which considers two major
perception tasks: maritime object detection and the more general maritime
obstacle segmentation. We present a new diverse maritime evaluation dataset
containing approximately 81k stereo images synchronized with an on-board IMU,
with over 60k objects annotated. We propose a new obstacle segmentation
performance evaluation protocol that reflects the detection accuracy in a way
meaningful for practical USV navigation. Nineteen recent state-of-the-art
object detection and obstacle segmentation methods are evaluated using the
proposed protocol, creating a benchmark to facilitate development of the field.
The proposed dataset, as well as evaluation routines, are made publicly
available at vicos.si/resources.
