  We propose a novel method that combines CycleGAN and inter-domain losses for
semi-supervised end-to-end automatic speech recognition. Inter-domain loss
targets the extraction of an intermediate shared representation of speech and
text inputs using a shared network. CycleGAN uses cycle-consistent loss and the
identity mapping loss to preserve relevant characteristics of the input feature
after converting from one domain to another. As such, both approaches are
suitable to train end-to-end models on unpaired speech-text inputs. In this
paper, we exploit the advantages from both inter-domain loss and CycleGAN to
achieve better shared representation of unpaired speech and text inputs and
thus improve the speech-to-text mapping. Our experimental results on the WSJ
eval92 and Voxforge (non English) show 8~8.5% character error rate reduction
over the baseline, and the results on LibriSpeech test_clean also show
noticeable improvement.
