  The lightweight MLP-based decoder has become increasingly promising for
semantic segmentation. However, the channel-wise MLP cannot expand the
receptive fields, lacking the context modeling capacity, which is critical to
semantic segmentation. In this paper, we propose a parametric-free patch rotate
operation to reorganize the pixels spatially. It first divides the feature map
into multiple groups and then rotates the patches within each group. Based on
the proposed patch rotate operation, we design a novel segmentation network,
named PRSeg, which includes an off-the-shelf backbone and a lightweight Patch
Rotate MLP decoder containing multiple Dynamic Patch Rotate Blocks
(DPR-Blocks). In each DPR-Block, the fully connected layer is performed
following a Patch Rotate Module (PRM) to exchange spatial information between
pixels. Specifically, in PRM, the feature map is first split into the reserved
part and rotated part along the channel dimension according to the predicted
probability of the Dynamic Channel Selection Module (DCSM), and our proposed
patch rotate operation is only performed on the rotated part. Extensive
experiments on ADE20K, Cityscapes and COCO-Stuff 10K datasets prove the
effectiveness of our approach. We expect that our PRSeg can promote the
development of MLP-based decoder in semantic segmentation.
