  Blind face restoration (BFR) is a challenging problem because of the
uncertainty of the degradation patterns. This paper proposes a Restoration with
Memorized Modulation (RMM) framework for universal BFR in diverse degraded
scenes and heterogeneous domains. We apply random noise as well as unsupervised
wavelet memory to adaptively modulate the face-enhancement generator,
considering attentional denormalization in both layer and instance levels.
Specifically, in the training stage, the low-level spatial feature embedding,
the wavelet memory embedding obtained by wavelet transformation of the
high-resolution image, as well as the disentangled high-level noise embeddings
are integrated, with the guidance of attentional maps generated from layer
normalization, instance normalization, and the original feature map. These
three embeddings are respectively associated with the spatial content,
high-frequency texture details, and a learnable universal prior against other
blind image degradation patterns. We store the spatial feature of the
low-resolution image and the corresponding wavelet style code as key and value
in the memory unit, respectively. In the test stage, the wavelet memory value
whose corresponding spatial key is the most matching with that of the inferred
image is retrieved to modulate the generator. Moreover, the universal prior
learned from the random noise has been memorized by training the modulation
network. Experimental results show the superiority of the proposed method
compared with the state-of-the-art methods, and a good generalization in the
wild.
