  Face restoration is important in face image processing, and has been widely
studied in recent years. However, previous works often fail to generate
plausible high quality (HQ) results for real-world low quality (LQ) face
images. In this paper, we propose a new progressive semantic-aware style
transformation framework, named PSFR-GAN, for face restoration. Specifically,
instead of using an encoder-decoder framework as previous methods, we formulate
the restoration of LQ face images as a multi-scale progressive restoration
procedure through semantic-aware style transformation. Given a pair of LQ face
image and its corresponding parsing map, we first generate a multi-scale
pyramid of the inputs, and then progressively modulate different scale features
from coarse-to-fine in a semantic-aware style transfer way. Compared with
previous networks, the proposed PSFR-GAN makes full use of the semantic
(parsing maps) and pixel (LQ images) space information from different scales of
input pairs. In addition, we further introduce a semantic aware style loss
which calculates the feature style loss for each semantic region individually
to improve the details of face textures. Finally, we pretrain a face parsing
network which can generate decent parsing maps from real-world LQ face images.
Experiment results show that our model trained with synthetic data can not only
produce more realistic high-resolution results for synthetic LQ inputs and but
also generalize better to natural LQ face images compared with state-of-the-art
methods. Codes are available at https://github.com/chaofengc/PSFRGAN.
