  We develop an automated video colorization framework that minimizes the
flickering of colors across frames. If we apply image colorization techniques
to successive frames of a video, they treat each frame as a separate
colorization task. Thus, they do not necessarily maintain the colors of a scene
consistently across subsequent frames. The proposed solution includes a novel
deep recurrent encoder-decoder architecture which is capable of maintaining
temporal and contextual coherence between consecutive frames of a video. We use
a high-level semantic feature extractor to automatically identify the context
of a scenario including objects, with a custom fusion layer that combines the
spatial and temporal features of a frame sequence. We demonstrate experimental
results, qualitatively showing that recurrent neural networks can be
successfully used to improve color consistency in video colorization.
