  Coastal water autonomous boats rely on robust perception methods for obstacle
detection and timely collision avoidance. The current state-of-the-art is based
on deep segmentation networks trained on large datasets. Per-pixel ground truth
labeling of such datasets, however, is labor-intensive and expensive. We
observe that far less information is required for practical obstacle avoidance
- the location of water edge on static obstacles like shore and approximate
location and bounds of dynamic obstacles in the water is sufficient to plan a
reaction. We propose a new scaffolding learning regime (SLR) that allows
training obstacle detection segmentation networks only from such weak
annotations, thus significantly reducing the cost of ground-truth labeling.
Experiments show that maritime obstacle segmentation networks trained using SLR
substantially outperform the same networks trained with dense ground truth
labels. Thus accuracy is not sacrificed for labelling simplicity but is in fact
improved, which is a remarkable result.
