  Speech enhancement in the time domain is becoming increasingly popular in
recent years, due to its capability to jointly enhance both the magnitude and
the phase of speech. In this work, we propose a dense convolutional network
(DCN) with self-attention for speech enhancement in the time domain. DCN is an
encoder and decoder based architecture with skip connections. Each layer in the
encoder and the decoder comprises a dense block and an attention module. Dense
blocks and attention modules help in feature extraction using a combination of
feature reuse, increased network depth, and maximum context aggregation.
Furthermore, we reveal previously unknown problems with a loss based on the
spectral magnitude of enhanced speech. To alleviate these problems, we propose
a novel loss based on magnitudes of enhanced speech and a predicted noise. Even
though the proposed loss is based on magnitudes only, a constraint imposed by
noise prediction ensures that the loss enhances both magnitude and phase.
Experimental results demonstrate that DCN trained with the proposed loss
substantially outperforms other state-of-the-art approaches to causal and
non-causal speech enhancement.
