  This paper investigates the problem of pseudo-healthy synthesis that is
defined as synthesizing a subject-specific pathology-free image from a
pathological one. Recent approaches based on Generative Adversarial Network
(GAN) have been developed for this task. However, these methods will inevitably
fall into the trade-off between preserving the subject-specific identity and
generating healthy-like appearances. To overcome this challenge, we propose a
novel adversarial training regime, Generator versus Segmentor (GVS), to
alleviate this trade-off by a divide-and-conquer strategy. We further consider
the deteriorating generalization performance of the segmentor throughout the
training and develop a pixel-wise weighted loss by muting the well-transformed
pixels to promote it. Moreover, we propose a new metric to measure how healthy
the synthetic images look. The qualitative and quantitative experiments on the
public dataset BraTS demonstrate that the proposed method outperforms the
existing methods. Besides, we also certify the effectiveness of our method on
datasets LiTS. Our implementation and pre-trained networks are publicly
available at https://github.com/Au3C2/Generator-Versus-Segmentor.
