  Panoptic image segmentation is the computer vision task of finding groups of
pixels in an image and assigning semantic classes and object instance
identifiers to them. Research in image segmentation has become increasingly
popular due to its critical applications in robotics and autonomous driving.
The research community thereby relies on publicly available benchmark dataset
to advance the state-of-the-art in computer vision. Due to the high costs of
densely labeling the images, however, there is a shortage of publicly available
ground truth labels that are suitable for panoptic segmentation. The high
labeling costs also make it challenging to extend existing datasets to the
video domain and to multi-camera setups. We therefore present the Waymo Open
Dataset: Panoramic Video Panoptic Segmentation Dataset, a large-scale dataset
that offers high-quality panoptic segmentation labels for autonomous driving.
We generate our dataset using the publicly available Waymo Open Dataset,
leveraging the diverse set of camera images. Our labels are consistent over
time for video processing and consistent across multiple cameras mounted on the
vehicles for full panoramic scene understanding. Specifically, we offer labels
for 28 semantic categories and 2,860 temporal sequences that were captured by
five cameras mounted on autonomous vehicles driving in three different
geographical locations, leading to a total of 100k labeled camera images. To
the best of our knowledge, this makes our dataset an order of magnitude larger
than existing datasets that offer video panoptic segmentation labels. We
further propose a new benchmark for Panoramic Video Panoptic Segmentation and
establish a number of strong baselines based on the DeepLab family of models.
We will make the benchmark and the code publicly available. Find the dataset at
https://waymo.com/open.
