  RGBD images with high quality annotations in the form of geometric (i.e.,
segmentation) and structural (i.e., how do the segments are mutually related in
3D) information provide valuable priors to a large number of scene and image
manipulation applications. While it is now simple to acquire RGBD images,
annotating them, automatically or manually, remains challenging especially in
cluttered noisy environments. We present SmartAnnotator, an interactive system
to facilitate annotating RGBD images. The system performs the tedious tasks of
grouping pixels, creating potential abstracted cuboids, inferring object
interactions in 3D, and comes up with various hypotheses. The user simply has
to flip through a list of suggestions for segment labels, finalize a selection,
and the system updates the remaining hypotheses. As objects are finalized, the
process speeds up with fewer ambiguities to resolve. Further, as more scenes
are annotated, the system makes better suggestions based on structural and
geometric priors learns from the previous annotation sessions. We test our
system on a large number of database scenes and report significant improvements
over naive low-level annotation tools.
