  Integrating a foreground object into a background scene with illumination
harmonization is an important but challenging task in computer vision and
augmented reality community. Existing methods mainly focus on foreground and
background appearance consistency or the foreground object shadow generation,
which rarely consider global appearance and illumination harmonization. In this
paper, we formulate seamless illumination harmonization as an illumination
exchange and aggregation problem. Specifically, we firstly apply a
physically-based rendering method to construct a large-scale, high-quality
dataset (named IH) for our task, which contains various types of foreground
objects and background scenes with different lighting conditions. Then, we
propose a deep image-based illumination harmonization GAN framework named
DIH-GAN, which makes full use of a multi-scale attention mechanism and
illumination exchange strategy to directly infer mapping relationship between
the inserted foreground object and the corresponding background scene.
Meanwhile, we also use adversarial learning strategy to further refine the
illumination harmonization result. Our method can not only achieve harmonious
appearance and illumination for the foreground object but also can generate
compelling shadow cast by the foreground object. Comprehensive experiments on
both our IH dataset and real-world images show that our proposed DIH-GAN
provides a practical and effective solution for image-based object illumination
harmonization editing, and validate the superiority of our method against
state-of-the-art methods. Our IH dataset is available at
https://github.com/zhongyunbao/Dataset.
