  Recent developments in the field of non-local attention (NLA) have led to a
renewed interest in self-similarity-based single image super-resolution (SISR).
Researchers usually used the NLA to explore non-local self-similarity (NSS) in
SISR and achieve satisfactory reconstruction results. However, a surprising
phenomenon that the reconstruction performance of the standard NLA is similar
to the NLA with randomly selected regions stimulated our interest to revisit
NLA. In this paper, we first analyzed the attention map of the standard NLA
from different perspectives and discovered that the resulting probability
distribution always has full support for every local feature, which implies a
statistical waste of assigning values to irrelevant non-local features,
especially for SISR which needs to model long-range dependence with a large
number of redundant non-local features. Based on these findings, we introduced
a concise yet effective soft thresholding operation to obtain
high-similarity-pass attention (HSPA), which is beneficial for generating a
more compact and interpretable distribution. Furthermore, we derived some key
properties of the soft thresholding operation that enable training our HSPA in
an end-to-end manner. The HSPA can be integrated into existing deep SISR models
as an efficient general building block. In addition, to demonstrate the
effectiveness of the HSPA, we constructed a deep high-similarity-pass attention
network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive
experimental results demonstrate that HSPAN outperforms state-of-the-art
approaches on both quantitative and qualitative evaluations.
