  Image harmonization aims to improve the quality of image compositing by
matching the "appearance" (\eg, color tone, brightness and contrast) between
foreground and background images. However, collecting large-scale annotated
datasets for this task requires complex professional retouching. Instead, we
propose a novel Self-Supervised Harmonization framework (SSH) that can be
trained using just "free" natural images without being edited. We reformulate
the image harmonization problem from a representation fusion perspective, which
separately processes the foreground and background examples, to address the
background occlusion issue. This framework design allows for a dual data
augmentation method, where diverse [foreground, background, pseudo GT] triplets
can be generated by cropping an image with perturbations using 3D color lookup
tables (LUTs). In addition, we build a real-world harmonization dataset as
carefully created by expert users, for evaluation and benchmarking purposes.
Our results show that the proposed self-supervised method outperforms previous
state-of-the-art methods in terms of reference metrics, visual quality, and
subject user study. Code and dataset are available at
\url{https://github.com/VITA-Group/SSHarmonization}.
