  In this work, we investigate performing semantic segmentation solely through
the training on image-sentence pairs. Due to the lack of dense annotations,
existing text-supervised methods can only learn to group an image into semantic
regions via pixel-insensitive feedback. As a result, their grouped results are
coarse and often contain small spurious regions, limiting the upper-bound
performance of segmentation. On the other hand, we observe that grouped results
from self-supervised models are more semantically consistent and break the
bottleneck of existing methods. Motivated by this, we introduce associate
self-supervised spatially-consistent grouping with text-supervised semantic
segmentation. Considering the part-like grouped results, we further adapt a
text-supervised model from image-level to region-level recognition with two
core designs. First, we encourage fine-grained alignment with a one-way
noun-to-region contrastive loss, which reduces the mismatched noun-region
pairs. Second, we adopt a contextually aware masking strategy to enable
simultaneous recognition of all grouped regions. Coupled with
spatially-consistent grouping and region-adapted recognition, our method
achieves 59.2% mIoU and 32.4% mIoU on Pascal VOC and Pascal Context benchmarks,
significantly surpassing the state-of-the-art methods.
