  In this paper, we propose the coarse-to-fine optimization for the task of
speech enhancement. Cosine similarity loss [1] has proven to be an effective
metric to measure similarity of speech signals. However, due to the large
variance of the enhanced speech with even the same cosine similarity loss in
high dimensional space, a deep neural network learnt with this loss might not
be able to predict enhanced speech with good quality. Our coarse-to-fine
strategy optimizes the cosine similarity loss for different granularities so
that more constraints are added to the prediction from high dimension to
relatively low dimension. In this way, the enhanced speech will better resemble
the clean speech. Experimental results show the effectiveness of our proposed
coarse-to-fine optimization in both discriminative models and generative
models. Moreover, we apply the coarse-to-fine strategy to the adversarial loss
in generative adversarial network (GAN) and propose dynamic perceptual loss,
which dynamically computes the adversarial loss from coarse resolution to fine
resolution. Dynamic perceptual loss further improves the accuracy and achieves
state-of-the-art results compared with other generative models.
