  We show that it is possible to learn semantic segmentation from very limited
amounts of manual annotations, by enforcing geometric 3D constraints between
multiple views. More exactly, image locations corresponding to the same
physical 3D point should all have the same label. We show that introducing such
constraints during learning is very effective, even when no manual label is
available for a 3D point, and can be done simply by employing techniques from
'general' semi-supervised learning to the context of semantic segmentation. To
demonstrate this idea, we use RGB-D image sequences of rigid scenes, for a
4-class segmentation problem derived from the ScanNet dataset. Starting from
RGB-D sequences with a few annotated frames, we show that we can incorporate
RGB-D sequences without any manual annotations to improve the performance,
which makes our approach very convenient. Furthermore, we demonstrate our
approach for semantic segmentation of objects on the LabelFusion dataset, where
we show that one manually labeled image in a scene is sufficient for high
performance on the whole scene.
