  Existing approaches for restoring weather-degraded images follow a
fully-supervised paradigm and they require paired data for training. However,
collecting paired data for weather degradations is extremely challenging, and
existing methods end up training on synthetic data. To overcome this issue, we
describe an approach for supervising deep networks that are based on CycleGAN,
thereby enabling the use of unlabeled real-world data for training.
Specifically, we introduce new losses for training CycleGAN that lead to more
effective training, resulting in high-quality reconstructions. These new losses
are obtained by jointly modeling the latent space embeddings of predicted clean
images and original clean images through Deep Gaussian Processes. This enables
the CycleGAN architecture to transfer the knowledge from one domain
(weather-degraded) to another (clean) more effectively. We demonstrate that the
proposed method can be effectively applied to different restoration tasks like
de-raining, de-hazing and de-snowing and it outperforms other unsupervised
techniques (that leverage weather-based characteristics) by a considerable
margin.
