  This work studies the joint rain and haze removal problem. In real-life
scenarios, rain and haze, two often co-occurring common weather phenomena, can
greatly degrade the clarity and quality of the scene images, leading to a
performance drop in the visual applications, such as autonomous driving.
However, jointly removing the rain and haze in scene images is ill-posed and
challenging, where the existence of haze and rain and the change of atmosphere
light, can both degrade the scene information. Current methods focus on the
contamination removal part, thus ignoring the restoration of the scene
information affected by the change of atmospheric light. We propose a novel
deep neural network, named Asymmetric Dual-decoder U-Net (ADU-Net), to address
the aforementioned challenge. The ADU-Net produces both the contamination
residual and the scene residual to efficiently remove the rain and haze while
preserving the fidelity of the scene information. Extensive experiments show
our work outperforms the existing state-of-the-art methods by a considerable
margin in both synthetic data and real-world data benchmarks, including
RainCityscapes, BID Rain, and SPA-Data. For instance, we improve the
state-of-the-art PSNR value by 2.26/4.57 on the RainCityscapes/SPA-Data,
respectively.
  Codes will be made available freely to the research community.
