  Contemporary speech enhancement predominantly relies on audio transforms that
are trained to reconstruct a clean speech waveform. The development of
high-performing neural network sound recognition systems has raised the
possibility of using deep feature representations as 'perceptual' losses with
which to train denoising systems. We explored their utility by first training
deep neural networks to classify either spoken words or environmental sounds
from audio. We then trained an audio transform to map noisy speech to an audio
waveform that minimized the difference in the deep feature representations
between the output audio and the corresponding clean audio. The resulting
transforms removed noise substantially better than baseline methods trained to
reconstruct clean waveforms, and also outperformed previous methods using deep
feature losses. However, a similar benefit was obtained simply by using losses
derived from the filter bank inputs to the deep networks. The results show that
deep features can guide speech enhancement, but suggest that they do not yet
outperform simple alternatives that do not involve learned features.
