  Effective shadow removal is pivotal in enhancing the visual quality of images
in various applications, ranging from computer vision to digital photography.
During the last decades physics and machine learning -based methodologies have
been proposed; however, most of them have limited capacity in capturing complex
shadow patterns due to restrictive model assumptions, neglecting the fact that
shadows usually appear at different scales. Also, current datasets used for
benchmarking shadow removal are composed of a limited number of images with
simple scenes containing mainly uniform shadows cast by single objects, whereas
only a few of them include both manual shadow annotations and paired
shadow-free images. Aiming to address all these limitations in the context of
natural scene imaging, including urban environments with complex scenes, the
contribution of this study is twofold: a) it proposes a novel deep learning
architecture, named Soft-Hard Attention U-net (SHAU), focusing on multiscale
shadow removal; b) it provides a novel synthetic dataset, named Multiscale
Shadow Removal Dataset (MSRD), containing complex shadow patterns of multiple
scales, aiming to serve as a privacy-preserving dataset for a more
comprehensive benchmarking of future shadow removal methodologies. Key
architectural components of SHAU are the soft and hard attention modules, which
along with multiscale feature extraction blocks enable effective shadow removal
of different scales and intensities. The results demonstrate the effectiveness
of SHAU over the relevant state-of-the-art shadow removal methods across
various benchmark datasets, improving the Peak Signal-to-Noise Ratio and Root
Mean Square Error for the shadow area by 25.1% and 61.3%, respectively.
