  For the task of speech enhancement, local learning objectives are agnostic to
phonetic structures helpful for speech recognition. We propose to add a global
criterion to ensure de-noised speech is useful for downstream tasks like ASR.
We first train a spectral classifier on clean speech to predict senone labels.
Then, the spectral classifier is joined with our speech enhancer as a noisy
speech recognizer. This model is taught to imitate the output of the spectral
classifier alone on clean speech. This \textit{mimic loss} is combined with the
traditional local criterion to train the speech enhancer to produce de-noised
speech. Feeding the de-noised speech to an off-the-shelf Kaldi training recipe
for the CHiME-2 corpus shows significant improvements in WER.
