  High-quality speech corpora are essential foundations for most speech
applications. However, such speech data are expensive and limited since they
are collected in professional recording environments. In this work, we propose
an encoder-decoder neural network to automatically enhance low-quality
recordings to professional high-quality recordings. To address channel
variability, we first filter out the channel characteristics from the original
input audio using the encoder network with adversarial training. Next, we
disentangle the channel factor from a reference audio. Conditioned on this
factor, an auto-regressive decoder is then used to predict the
target-environment Mel spectrogram. Finally, we apply a neural vocoder to
synthesize the speech waveform. Experimental results show that the proposed
system can generate a professional high-quality speech waveform when setting
high-quality audio as the reference. It also improves speech enhancement
performance compared with several state-of-the-art baseline systems.
