  This paper investigates into the colorization problem which converts a
grayscale image to a colorful version. This is a very difficult problem and
normally requires manual adjustment to achieve artifact-free quality. For
instance, it normally requires human-labelled color scribbles on the grayscale
target image or a careful selection of colorful reference images (e.g.,
capturing the same scene in the grayscale target image). Unlike the previous
methods, this paper aims at a high-quality fully-automatic colorization method.
With the assumption of a perfect patch matching technique, the use of an
extremely large-scale reference database (that contains sufficient color
images) is the most reliable solution to the colorization problem. However,
patch matching noise will increase with respect to the size of the reference
database in practice. Inspired by the recent success in deep learning
techniques which provide amazing modeling of large-scale data, this paper
re-formulates the colorization problem so that deep learning techniques can be
directly employed. To ensure artifact-free quality, a joint bilateral filtering
based post-processing step is proposed. We further develop an adaptive image
clustering technique to incorporate the global image information. Numerous
experiments demonstrate that our method outperforms the state-of-art algorithms
both in terms of quality and speed.
