  This paper presents the first end-to-end network for exemplar-based video
colorization. The main challenge is to achieve temporal consistency while
remaining faithful to the reference style. To address this issue, we introduce
a recurrent framework that unifies the semantic correspondence and color
propagation steps. Both steps allow a provided reference image to guide the
colorization of every frame, thus reducing accumulated propagation errors.
Video frames are colorized in sequence based on the colorization history, and
its coherency is further enforced by the temporal consistency loss. All of
these components, learned end-to-end, help produce realistic videos with good
temporal stability. Experiments show our result is superior to the
state-of-the-art methods both quantitatively and qualitatively.
