  The recently developed transformer networks have achieved impressive
performance in image denoising by exploiting the self-attention (SA) in images.
However, the existing methods mostly use a relatively small window to compute
SA due to the quadratic complexity of it, which limits the model's ability to
model long-term image information. In this paper, we propose the
spatial-frequency attention network (SFANet) to enhance the network's ability
in exploiting long-range dependency. For spatial attention module (SAM), we
adopt dilated SA to model long-range dependency. In the frequency attention
module (FAM), we exploit more global information by using Fast Fourier
Transform (FFT) by designing a window-based frequency channel attention (WFCA)
block to effectively model deep frequency features and their dependencies. To
make our module applicable to images of different sizes and keep the model
consistency between training and inference, we apply window-based FFT with a
set of fixed window sizes. In addition, channel attention is computed on both
real and imaginary parts of the Fourier spectrum, which further improves
restoration performance. The proposed WFCA block can effectively model image
long-range dependency with acceptable complexity. Experiments on multiple
denoising benchmarks demonstrate the leading performance of SFANet network.
