  It is a time-consuming and tedious work for manually colorizing anime line
drawing images, which is an essential stage in cartoon animation creation
pipeline. Reference-based line drawing colorization is a challenging task that
relies on the precise cross-domain long-range dependency modelling between the
line drawing and reference image. Existing learning methods still utilize
generative adversarial networks (GANs) as one key module of their model
architecture. In this paper, we propose a novel method called AnimeDiffusion
using diffusion models that performs anime face line drawing colorization
automatically. To the best of our knowledge, this is the first diffusion model
tailored for anime content creation. In order to solve the huge training
consumption problem of diffusion models, we design a hybrid training strategy,
first pre-training a diffusion model with classifier-free guidance and then
fine-tuning it with image reconstruction guidance. We find that with a few
iterations of fine-tuning, the model shows wonderful colorization performance,
as illustrated in Fig. 1. For training AnimeDiffusion, we conduct an anime face
line drawing colorization benchmark dataset, which contains 31696 training data
and 579 testing data. We hope this dataset can fill the gap of no available
high resolution anime face dataset for colorization method evaluation. Through
multiple quantitative metrics evaluated on our dataset and a user study, we
demonstrate AnimeDiffusion outperforms state-of-the-art GANs-based models for
anime face line drawing colorization. We also collaborate with professional
artists to test and apply our AnimeDiffusion for their creation work. We
release our code on https://github.com/xq-meng/AnimeDiffusion.
