  Illumination effects in images, specifically cast shadows and shading, have
been shown to decrease the performance of deep neural networks on a large
number of vision-based detection, recognition and segmentation tasks in urban
driving scenes. A key factor that contributes to this performance gap is the
lack of `time-of-day' diversity within real, labeled datasets. There have been
impressive advances in the realm of image to image translation in transferring
previously unseen visual effects into a dataset, specifically in day to night
translation. However, it is not easy to constrain what visual effects, let
alone illumination effects, are transferred from one dataset to another during
the training process. To address this problem, we propose deep learning
framework, called Shadow Transfer, that can relight complex outdoor scenes by
transferring realistic shadow, shading, and other lighting effects onto a
single image. The novelty of the proposed framework is that it is both
self-supervised, and is designed to operate on sensor and label information
that is easily available in autonomous vehicle datasets. We show the
effectiveness of this method on both synthetic and real datasets, and we
provide experiments that demonstrate that the proposed method produces images
of higher visual quality than state of the art image to image translation
methods.
