  Restoring the clean background from the superimposed images containing a
noisy layer is the common crux of a classical category of tasks on image
restoration such as image reflection removal, image deraining and image
dehazing. These tasks are typically formulated and tackled individually due to
the diverse and complicated appearance patterns of noise layers within the
image. In this work we present the Deep-Masking Generative Network (DMGN),
which is a unified framework for background restoration from the superimposed
images and is able to cope with different types of noise. Our proposed DMGN
follows a coarse-to-fine generative process: a coarse background image and a
noise image are first generated in parallel, then the noise image is further
leveraged to refine the background image to achieve a higher-quality background
image. In particular, we design the novel Residual Deep-Masking Cell as the
core operating unit for our DMGN to enhance the effective information and
suppress the negative information during image generation via learning a gating
mask to control the information flow. By iteratively employing this Residual
Deep-Masking Cell, our proposed DMGN is able to generate both high-quality
background image and noisy image progressively. Furthermore, we propose a
two-pronged strategy to effectively leverage the generated noise image as
contrasting cues to facilitate the refinement of the background image.
Extensive experiments across three typical tasks for image background
restoration, including image reflection removal, image rain steak removal and
image dehazing, show that our DMGN consistently outperforms state-of-the-art
methods specifically designed for each single task.
