  This paper presents a new mechanism to facilitate the training of mask
transformers for efficient panoptic segmentation, democratizing its deployment.
We observe that due to its high complexity, the training objective of panoptic
segmentation will inevitably lead to much higher false positive penalization.
Such unbalanced loss makes the training process of the end-to-end
mask-transformer based architectures difficult, especially for efficient
models. In this paper, we present ReMaX that adds relaxation to mask
predictions and class predictions during training for panoptic segmentation. We
demonstrate that via these simple relaxation techniques during training, our
model can be consistently improved by a clear margin \textbf{without} any extra
computational cost on inference. By combining our method with efficient
backbones like MobileNetV3-Small, our method achieves new state-of-the-art
results for efficient panoptic segmentation on COCO, ADE20K and Cityscapes.
Code and pre-trained checkpoints will be available at
\url{https://github.com/google-research/deeplab2}.
