  We develop a fully automatic image colorization system. Our approach
leverages recent advances in deep networks, exploiting both low-level and
semantic representations. As many scene elements naturally appear according to
multimodal color distributions, we train our model to predict per-pixel color
histograms. This intermediate output can be used to automatically generate a
color image, or further manipulated prior to image formation. On both fully and
partially automatic colorization tasks, we outperform existing methods. We also
explore colorization as a vehicle for self-supervised visual representation
learning.
