  Automatic animation line art colorization is a challenging computer vision
problem, since the information of the line art is highly sparse and abstracted
and there exists a strict requirement for the color and style consistency
between frames. Recently, a lot of Generative Adversarial Network (GAN) based
image-to-image translation methods for single line art colorization have
emerged. They can generate perceptually appealing results conditioned on line
art images. However, these methods can not be adopted for the purpose of
animation colorization because there is a lack of consideration of the
in-between frame consistency. Existing methods simply input the previous
colored frame as a reference to color the next line art, which will mislead the
colorization due to the spatial misalignment of the previous colored frame and
the next line art especially at positions where apparent changes happen. To
address these challenges, we design a kind of correlation matching feature
transfer model (called CMFT) to align the colored reference feature in a
learnable way and integrate the model into an U-Net based generator in a
coarse-to-fine manner. This enables the generator to transfer the layer-wise
synchronized features from the deep semantic code to the content progressively.
Extension evaluation shows that CMFT model can effectively improve the
in-between consistency and the quality of colored frames especially when the
motion is intense and diverse.
