  Current image harmonization methods consider the entire background as the
guidance for harmonization. However, this may limit the capability for user to
choose any specific object/person in the background to guide the harmonization.
To enable flexible interaction between user and harmonization, we introduce
interactive harmonization, a new setting where the harmonization is performed
with respect to a selected \emph{region} in the reference image instead of the
entire background. A new flexible framework that allows users to pick certain
regions of the background image and use it to guide the harmonization is
proposed. Inspired by professional portrait harmonization users, we also
introduce a new luminance matching loss to optimally match the color/luminance
conditions between the composite foreground and select reference region. This
framework provides more control to the image harmonization pipeline achieving
visually pleasing portrait edits. Furthermore, we also introduce a new dataset
carefully curated for validating portrait harmonization. Extensive experiments
on both synthetic and real-world datasets show that the proposed approach is
efficient and robust compared to previous harmonization baselines, especially
for portraits. Project Webpage at
\href{https://jeya-maria-jose.github.io/IPH-web/}{https://jeya-maria-jose.github.io/IPH-web/}
