  Despite the strong modeling power of neural network acoustic models, speech
enhancement has been shown to deliver additional word error rate improvements
if multi-channel data is available. However, there has been a longstanding
debate whether enhancement should also be carried out on the ASR training data.
In an extensive experimental evaluation on the acoustically very challenging
CHiME-5 dinner party data we show that: (i) cleaning up the training data can
lead to substantial error rate reductions, and (ii) enhancement in training is
advisable as long as enhancement in test is at least as strong as in training.
This approach stands in contrast and delivers larger gains than the common
strategy reported in the literature to augment the training database with
additional artificially degraded speech. Together with an acoustic model
topology consisting of initial CNN layers followed by factorized TDNN layers we
achieve with 41.6% and 43.2% WER on the DEV and EVAL test sets, respectively, a
new single-system state-of-the-art result on the CHiME-5 data. This is a 8%
relative improvement compared to the best word error rate published so far for
a speech recognizer without system combination.
