  We present a fully automatic approach to video colorization with
self-regularization and diversity. Our model contains a colorization network
for video frame colorization and a refinement network for spatiotemporal color
refinement. Without any labeled data, both networks can be trained with
self-regularized losses defined in bilateral and temporal space. The bilateral
loss enforces color consistency between neighboring pixels in a bilateral space
and the temporal loss imposes constraints between corresponding pixels in two
nearby frames. While video colorization is a multi-modal problem, our method
uses a perceptual loss with diversity to differentiate various modes in the
solution space. Perceptual experiments demonstrate that our approach
outperforms state-of-the-art approaches on fully automatic video colorization.
The results are shown in the supplementary video at
https://youtu.be/Y15uv2jnK-4
