  Recent advances in pixel-level tasks (e.g. segmentation) illustrate the
benefit of of long-range interactions between aggregated region-based
representations that can enhance local features. However, such aggregated
representations, often in the form of attention, fail to model the underlying
semantics of the scene (e.g. individual objects and, by extension, their
interactions). In this work, we address the issue by proposing a component that
learns to project image features into latent representations and reason between
them using a transformer encoder to generate contextualized and
scene-consistent representations which are fused with original image features.
Our design encourages the latent regions to represent semantic concepts by
ensuring that the activated regions are spatially disjoint and the union of
such regions corresponds to a connected object segment. The proposed semantic
global reasoning (SGR) component is end-to-end trainable and can be easily
added to a wide variety of backbones (CNN or transformer-based) and
segmentation heads (per-pixel or mask classification) to consistently improve
the segmentation results on different datasets. In addition, our latent tokens
are semantically interpretable and diverse and provide a rich set of features
that can be transferred to downstream tasks like object detection and
segmentation, with improved performance. Furthermore, we also proposed metrics
to quantify the semantics of latent tokens at both class \& instance level.
