  Convolutional neural network (CNN) have proven its success for semantic
segmentation, which is a core task of emerging industrial applications such as
autonomous driving. However, most progress in semantic segmentation of urban
scenes is reported on standard scenarios, i.e., daytime scenes with favorable
illumination conditions. In practical applications, the outdoor weather and
illumination are changeable, e.g., cloudy and nighttime, which results in a
significant drop of semantic segmentation accuracy of CNN only trained with
daytime data. In this paper, we propose a novel generative adversarial network
(namely Mutual-GAN) to alleviate the accuracy decline when daytime-trained
neural network is applied to videos captured under adverse weather conditions.
The proposed Mutual-GAN adopts mutual information constraint to preserve
image-objects during cross-weather adaptation, which is an unsolved problem for
most unsupervised image-to-image translation approaches (e.g., CycleGAN). The
proposed Mutual-GAN is evaluated on two publicly available driving video
datasets (i.e., CamVid and SYNTHIA). The experimental results demonstrate that
our Mutual-GAN can yield visually plausible translated images and significantly
improve the semantic segmentation accuracy of daytime-trained deep learning
network while processing videos under challenging weathers.
