  Large-scale deployment of fully autonomous vehicles requires a very high
degree of robustness to unstructured traffic, and weather conditions, and
should prevent unsafe mispredictions. While there are several datasets and
benchmarks focusing on segmentation for drive scenes, they are not specifically
focused on safety and robustness issues. We introduce the IDD-AW dataset, which
provides 5000 pairs of high-quality images with pixel-level annotations,
captured under rain, fog, low light, and snow in unstructured driving
conditions. As compared to other adverse weather datasets, we provide i.) more
annotated images, ii.) paired Near-Infrared (NIR) image for each frame, iii.)
larger label set with a 4-level label hierarchy to capture unstructured traffic
conditions. We benchmark state-of-the-art models for semantic segmentation in
IDD-AW. We also propose a new metric called ''Safe mean Intersection over Union
(Safe mIoU)'' for hierarchical datasets which penalizes dangerous
mispredictions that are not captured in the traditional definition of mean
Intersection over Union (mIoU). The results show that IDD-AW is one of the most
challenging datasets to date for these tasks. The dataset and code will be
available here: http://iddaw.github.io.
