  Recently, deep learning-based Image-to-Image (I2I) networks have become the
predominant choice for I2I tasks such as image super-resolution and denoising.
Despite their remarkable performance, the backdoor vulnerability of I2I
networks has not been explored. To fill this research gap, we conduct a
comprehensive investigation on the susceptibility of I2I networks to backdoor
attacks. Specifically, we propose a novel backdoor attack technique, where the
compromised I2I network behaves normally on clean input images, yet outputs a
predefined image of the adversary for malicious input images containing the
trigger. To achieve this I2I backdoor attack, we propose a targeted universal
adversarial perturbation (UAP) generation algorithm for I2I networks, where the
generated UAP is used as the backdoor trigger. Additionally, in the backdoor
training process that contains the main task and the backdoor task, multi-task
learning (MTL) with dynamic weighting methods is employed to accelerate
convergence rates. In addition to attacking I2I tasks, we extend our I2I
backdoor to attack downstream tasks, including image classification and object
detection. Extensive experiments demonstrate the effectiveness of the I2I
backdoor on state-of-the-art I2I network architectures, as well as the
robustness against different mainstream backdoor defenses.
