  Semantic segmentation based on sparse annotation has advanced in recent
years. It labels only part of each object in the image, leaving the remainder
unlabeled. Most of the existing approaches are time-consuming and often
necessitate a multi-stage training strategy. In this work, we propose a simple
yet effective sparse annotated semantic segmentation framework based on
segformer, dubbed SASFormer, that achieves remarkable performance.
Specifically, the framework first generates hierarchical patch attention maps,
which are then multiplied by the network predictions to produce correlated
regions separated by valid labels. Besides, we also introduce the affinity loss
to ensure consistency between the features of correlation results and network
predictions. Extensive experiments showcase that our proposed approach is
superior to existing methods and achieves cutting-edge performance. The source
code is available at \url{https://github.com/su-hui-zz/SASFormer}.
