  Developing robot perception systems for recognizing objects in the real-world
requires computer vision algorithms to be carefully scrutinized with respect to
the expected operating domain. This demands large quantities of ground truth
data to rigorously evaluate the performance of algorithms. This paper presents
the EasyLabel tool for easily acquiring high quality ground truth annotation of
objects at the pixel-level in densely cluttered scenes. In a semi-automatic
process, complex scenes are incrementally built and EasyLabel exploits depth
change to extract precise object masks at each step. We use this tool to
generate the Object Cluttered Indoor Dataset (OCID) that captures diverse
settings of objects, background, context, sensor to scene distance, viewpoint
angle and lighting conditions. OCID is used to perform a systematic comparison
of existing object segmentation methods. The baseline comparison supports the
need for pixel- and object-wise annotation to progress robot vision towards
realistic applications. This insight reveals the usefulness of EasyLabel and
OCID to better understand the challenges that robots face in the real-world.
  Copyright 20XX IEEE. Personal use of this material is permitted. Permission
from IEEE must be obtained for all other uses, in any current or future media,
including reprinting/republishing this material for advertising or promotional
purposes, creating new collective works, for resale or redistribution to
servers or lists, or reuse of any copyrighted component of this work in other
works.
