  Most studies on speech enhancement generally don't consider the energy
distribution of speech in time-frequency (T-F) representation, which is
important for accurate prediction of mask or spectra. In this paper, we present
a simple yet effective T-F attention (TFA) module, where a 2-D attention map is
produced to provide differentiated weights to the spectral components of T-F
representation. To validate the effectiveness of our proposed TFA module, we
use the residual temporal convolution network (ResTCN) as the backbone network
and conduct extensive experiments on two commonly used training targets. Our
experiments demonstrate that applying our TFA module significantly improves the
performance in terms of five objective evaluation metrics with negligible
parameter overhead. The evaluation results show that the proposed ResTCN with
the TFA module (ResTCN+TFA) consistently outperforms other baselines by a large
margin.
