  The non-local network has become a widely used technique for semantic
segmentation, which computes an attention map to measure the relationships of
each pixel pair. However, most of the current popular non-local models tend to
ignore the phenomenon that the calculated attention map appears to be very
noisy, containing inter-class and intra-class inconsistencies, which lowers the
accuracy and reliability of the non-local methods. In this paper, we
figuratively denote these inconsistencies as attention noises and explore the
solutions to denoise them. Specifically, we inventively propose a Denoised
Non-Local Network (Denoised NL), which consists of two primary modules, i.e.,
the Global Rectifying (GR) block and the Local Retention (LR) block, to
eliminate the inter-class and intra-class noises respectively. First, GR adopts
the class-level predictions to capture a binary map to distinguish whether the
selected two pixels belong to the same category. Second, LR captures the
ignored local dependencies and further uses them to rectify the unwanted
hollows in the attention map. The experimental results on two challenging
semantic segmentation datasets demonstrate the superior performance of our
model. Without any external training data, our proposed Denoised NL can achieve
the state-of-the-art performance of 83.5\% and 46.69\% mIoU on Cityscapes and
ADE20K, respectively.
