  Unpaired image-to-image translation (I2I) is an ill-posed problem, as an
infinite number of translation functions can map the source domain distribution
to the target distribution. Therefore, much effort has been put into designing
suitable constraints, e.g., cycle consistency (CycleGAN), geometry consistency
(GCGAN), and contrastive learning-based constraints (CUTGAN), that help better
pose the problem. However, these well-known constraints have limitations: (1)
they are either too restrictive or too weak for specific I2I tasks; (2) these
methods result in content distortion when there is a significant spatial
variation between the source and target domains. This paper proposes a
universal regularization technique called maximum spatial perturbation
consistency (MSPC), which enforces a spatial perturbation function (T ) and the
translation operator (G) to be commutative (i.e., TG = GT ). In addition, we
introduce two adversarial training components for learning the spatial
perturbation function. The first one lets T compete with G to achieve maximum
perturbation. The second one lets G and T compete with discriminators to align
the spatial variations caused by the change of object size, object distortion,
background interruptions, etc. Our method outperforms the state-of-the-art
methods on most I2I benchmarks. We also introduce a new benchmark, namely the
front face to profile face dataset, to emphasize the underlying challenges of
I2I for real-world applications. We finally perform ablation experiments to
study the sensitivity of our method to the severity of spatial perturbation and
its effectiveness for distribution alignment.
