  Most methods for object instance segmentation require all training examples
to be labeled with segmentation masks. This requirement makes it expensive to
annotate new categories and has restricted instance segmentation models to ~100
well-annotated classes. The goal of this paper is to propose a new partially
supervised training paradigm, together with a novel weight transfer function,
that enables training instance segmentation models on a large set of categories
all of which have box annotations, but only a small fraction of which have mask
annotations. These contributions allow us to train Mask R-CNN to detect and
segment 3000 visual concepts using box annotations from the Visual Genome
dataset and mask annotations from the 80 classes in the COCO dataset. We
evaluate our approach in a controlled study on the COCO dataset. This work is a
first step towards instance segmentation models that have broad comprehension
of the visual world.
