  Image colorization is inherently an ill-posed problem with multi-modal
uncertainty. Previous methods leverage the deep neural network to map input
grayscale images to plausible color outputs directly. Although these
learning-based methods have shown impressive performance, they usually fail on
the input images that contain multiple objects. The leading cause is that
existing models perform learning and colorization on the entire image. In the
absence of a clear figure-ground separation, these models cannot effectively
locate and learn meaningful object-level semantics. In this paper, we propose a
method for achieving instance-aware colorization. Our network architecture
leverages an off-the-shelf object detector to obtain cropped object images and
uses an instance colorization network to extract object-level features. We use
a similar network to extract the full-image features and apply a fusion module
to full object-level and image-level features to predict the final colors. Both
colorization networks and fusion modules are learned from a large-scale
dataset. Experimental results show that our work outperforms existing methods
on different quality metrics and achieves state-of-the-art performance on image
colorization.
