  Fully-supervised shadow removal methods achieve the best restoration
qualities on public datasets but still generate some shadow remnants. One of
the reasons is the lack of large-scale shadow & shadow-free image pairs.
Unsupervised methods can alleviate the issue but their restoration qualities
are much lower than those of fully-supervised methods. In this work, we find
that pretraining shadow removal networks on the image inpainting dataset can
reduce the shadow remnants significantly: a naive encoder-decoder network gets
competitive restoration quality w.r.t. the state-of-the-art methods via only
10% shadow & shadow-free image pairs. After analyzing networks with/without
inpainting pre-training via the information stored in the weight (IIW), we find
that inpainting pretraining improves restoration quality in non-shadow regions
and enhances the generalization ability of networks significantly.
Additionally, shadow removal fine-tuning enables networks to fill in the
details of shadow regions. Inspired by these observations we formulate shadow
removal as an adaptive fusion task that takes advantage of both shadow removal
and image inpainting. Specifically, we develop an adaptive fusion network
consisting of two encoders, an adaptive fusion block, and a decoder. The two
encoders are responsible for extracting the feature from the shadow image and
the shadow-masked image respectively. The adaptive fusion block is responsible
for combining these features in an adaptive manner. Finally, the decoder
converts the adaptive fused features to the desired shadow-free result. The
extensive experiments show that our method empowered with inpainting
outperforms all state-of-the-art methods.
