  Exploiting multi-scale features has shown great potential in tackling
semantic segmentation problems. The aggregation is commonly done with sum or
concatenation (concat) followed by convolutional (conv) layers. However, it
fully passes down the high-level context to the following hierarchy without
considering their interrelation. In this work, we aim to enable the low-level
feature to aggregate the complementary context from adjacent high-level feature
maps by a cross-scale pixel-to-region relation operation. We leverage
cross-scale context propagation to make the long-range dependency capturable
even by the high-resolution low-level features. To this end, we employ an
efficient feature pyramid network to obtain multi-scale features. We propose a
Relational Semantics Extractor (RSE) and Relational Semantics Propagator (RSP)
for context extraction and propagation respectively. Then we stack several RSP
into an RSP head to achieve the progressive top-down distribution of the
context. Experiment results on two challenging datasets Cityscapes and COCO
demonstrate that the RSP head performs competitively on both semantic
segmentation and panoptic segmentation with high efficiency. It outperforms
DeeplabV3 [1] by 0.7% with 75% fewer FLOPs (multiply-adds) in the semantic
segmentation task.
