  Robust maritime obstacle detection is critical for safe navigation of
autonomous boats and timely collision avoidance. The current state-of-the-art
is based on deep segmentation networks trained on large datasets. However,
per-pixel ground truth labeling of such datasets is labor-intensive and
expensive. We propose a new scaffolding learning regime (SLR) that leverages
weak annotations consisting of water edges, the horizon location, and obstacle
bounding boxes to train segmentation-based obstacle detection networks, thereby
reducing the required ground truth labeling effort by a factor of twenty. SLR
trains an initial model from weak annotations and then alternates between
re-estimating the segmentation pseudo-labels and improving the network
parameters. Experiments show that maritime obstacle segmentation networks
trained using SLR on weak annotations not only match but outperform the same
networks trained with dense ground truth labels, which is a remarkable result.
In addition to the increased accuracy, SLR also increases domain generalization
and can be used for domain adaptation with a low manual annotation load. The
SLR code and pre-trained models are available at
https://github.com/lojzezust/SLR .
