  This study investigates the vulnerability of semantic segmentation models to
adversarial input perturbations, in the domain of off-road autonomous driving.
Despite good performance in generic conditions, the state-of-the-art
classifiers are often susceptible to (even) small perturbations, ultimately
resulting in inaccurate predictions with high confidence. Prior research has
directed their focus on making models more robust by modifying the architecture
and training with noisy input images, but has not explored the influence of
datasets in adversarial attacks. Our study aims to address this gap by
examining the impact of non-robust features in off-road datasets and comparing
the effects of adversarial attacks on different segmentation network
architectures. To enable this, a robust dataset is created consisting of only
robust features and training the networks on this robustified dataset. We
present both qualitative and quantitative analysis of our findings, which have
important implications on improving the robustness of machine learning models
in off-road autonomous driving applications. Additionally, this work
contributes to the safe navigation of autonomous robot Unimog U5023 in rough
off-road unstructured environments by evaluating the robustness of segmentation
outputs. The code is publicly available at
https://github.com/rohtkumar/adversarial_attacks_ on_segmentation
