  We propose a real-time general purpose semantic segmentation architecture,
RGPNet, which achieves significant performance gain in complex environments.
RGPNet consists of a light-weight asymmetric encoder-decoder and an adaptor.
The adaptor helps preserve and refine the abstract concepts from multiple
levels of distributed representations between the encoder and decoder. It also
facilitates the gradient flow from deeper layers to shallower layers. Our
experiments demonstrate that RGPNet can generate segmentation results in
real-time with comparable accuracy to the state-of-the-art non-real-time heavy
models. Moreover, towards green AI, we show that using an optimized
label-relaxation technique with progressive resizing can reduce the training
time by up to 60% while preserving the performance. We conclude that RGPNet
obtains a better speed-accuracy trade-off across multiple datasets.
