  Instance segmentation models today are very accurate when trained on large
annotated datasets, but collecting mask annotations at scale is prohibitively
expensive. We address the partially supervised instance segmentation problem in
which one can train on (significantly cheaper) bounding boxes for all
categories but use masks only for a subset of categories. In this work, we
focus on a popular family of models which apply differentiable cropping to a
feature map and predict a mask based on the resulting crop. Under this family,
we study Mask R-CNN and discover that instead of its default strategy of
training the mask-head with a combination of proposals and groundtruth boxes,
training the mask-head with only groundtruth boxes dramatically improves its
performance on novel classes. This training strategy also allows us to take
advantage of alternative mask-head architectures, which we exploit by replacing
the typical mask-head of 2-4 layers with significantly deeper off-the-shelf
architectures (e.g. ResNet, Hourglass models). While many of these
architectures perform similarly when trained in fully supervised mode, our main
finding is that they can generalize to novel classes in dramatically different
ways. We call this ability of mask-heads to generalize to unseen classes the
strong mask generalization effect and show that without any specialty modules
or losses, we can achieve state-of-the-art results in the partially supervised
COCO instance segmentation benchmark. Finally, we demonstrate that our effect
is general, holding across underlying detection methodologies (including
anchor-based, anchor-free or no detector at all) and across different backbone
networks. Code and pre-trained models are available at https://git.io/deepmac.
