  Improving subjective sound quality of enhanced signals is one of the most
important missions in speech enhancement. For evaluating the subjective
quality, several methods related to perceptually-motivated objective sound
quality assessment (OSQA) have been proposed such as PESQ (perceptual
evaluation of speech quality). However, direct use of such measures for
training deep neural network (DNN) is not allowed in most cases because popular
OSQAs are non-differentiable with respect to DNN parameters. Therefore, the
previous study has proposed to approximate the score of OSQAs by an auxiliary
DNN so that its gradient can be used for training the primary DNN. One problem
with this approach is instability of the training caused by the approximation
error of the score. To overcome this problem, we propose to use stabilization
techniques borrowed from reinforcement learning. The experiments, aimed to
increase the score of PESQ as an example, show that the proposed method (i) can
stably train a DNN to increase PESQ, (ii) achieved the state-of-the-art PESQ
score on a public dataset, and (iii) resulted in better sound quality than
conventional methods based on subjective evaluation.
