  A surge of interest has emerged in weakly supervised semantic segmentation
due to its remarkable efficiency in recent years. Existing approaches based on
transformers mainly focus on exploring the affinity matrix to boost CAMs with
global relationships. While in this work, we first perform a scrupulous
examination towards the impact of successive affinity matrices and discover
that they possess an inclination toward sparsification as the network
approaches convergence, hence disclosing a manifestation of over-smoothing.
Besides, it has been observed that enhanced attention maps tend to evince a
substantial amount of extraneous background noise in deeper layers. Drawing
upon this, we posit a daring conjecture that the undisciplined over-smoothing
phenomenon introduces a noteworthy quantity of semantically irrelevant
background noise, causing performance degradation. To alleviate this issue, we
propose a novel perspective that highlights the objects of interest by
investigating the regions of the trait, thereby fostering an extensive
comprehension of the successive affinity matrix. Consequently, we suggest an
adaptive re-activation mechanism (AReAM) that alleviates the issue of
incomplete attention within the object and the unbounded background noise.
AReAM accomplishes this by supervising high-level attention with shallow
affinity matrices, yielding promising results. Exhaustive experiments conducted
on the commonly used dataset manifest that segmentation results can be greatly
improved through our proposed AReAM, which imposes restrictions on each
affinity matrix in deep layers to make it attentive to semantic regions.
