  We introduce MGNet, a multi-task framework for monocular geometric scene
understanding. We define monocular geometric scene understanding as the
combination of two known tasks: Panoptic segmentation and self-supervised
monocular depth estimation. Panoptic segmentation captures the full scene not
only semantically, but also on an instance basis. Self-supervised monocular
depth estimation uses geometric constraints derived from the camera measurement
model in order to measure depth from monocular video sequences only. To the
best of our knowledge, we are the first to propose the combination of these two
tasks in one single model. Our model is designed with focus on low latency to
provide fast inference in real-time on a single consumer-grade GPU. During
deployment, our model produces dense 3D point clouds with instance aware
semantic labels from single high-resolution camera images. We evaluate our
model on two popular autonomous driving benchmarks, i.e., Cityscapes and KITTI,
and show competitive performance among other real-time capable methods. Source
code is available at https://github.com/markusschoen/MGNet.
