  Generating artistic portraits is a challenging problem in computer vision.
Existing portrait stylization models that generate good quality results are
based on Image-to-Image Translation and require abundant data from both source
and target domains. However, without enough data, these methods would result in
overfitting. In this work, we propose CtlGAN, a new few-shot artistic portraits
generation model with a novel contrastive transfer learning strategy. We adapt
a pretrained StyleGAN in the source domain to a target artistic domain with no
more than 10 artistic faces. To reduce overfitting to the few training
examples, we introduce a novel Cross-Domain Triplet loss which explicitly
encourages the target instances generated from different latent codes to be
distinguishable. We propose a new encoder which embeds real faces into Z+ space
and proposes a dual-path training strategy to better cope with the adapted
decoder and eliminate the artifacts. Extensive qualitative, quantitative
comparisons and a user study show our method significantly outperforms
state-of-the-arts under 10-shot and 1-shot settings and generates high quality
artistic portraits. The code will be made publicly available.
