  This paper focuses on using voice conversion (VC) to improve the speech
intelligibility of surgical patients who have had parts of their articulators
removed. Due to the difficulty of data collection, VC without parallel data is
highly desired. Although techniques for unparallel VC, for example, CycleGAN,
have been developed, they usually focus on transforming the speaker identity,
and directly transforming the speech of one speaker to that of another speaker
and as such do not address the task here. In this paper, we propose a new
approach for unparallel VC. The proposed approach transforms impaired speech to
normal speech while preserving the linguistic content and speaker
characteristics. To our knowledge, this is the first end-to-end GAN-based
unsupervised VC model applied to impaired speech. The experimental results show
that the proposed approach outperforms CycleGAN.
