  A deep neural network (DNN)-based speech enhancement (SE) aiming to maximize
the performance of an automatic speech recognition (ASR) system is proposed in
this paper. In order to optimize the DNN-based SE model in terms of the
character error rate (CER), which is one of the metric to evaluate the ASR
system and generally non-differentiable, our method uses two DNNs: one for
speech processing and one for mimicking the output CERs derived through an
acoustic model (AM). Then both of DNNs are alternately optimized in the
training phase. Even if the AM is a black-box, e.g., like one provided by a
third-party, the proposed method enables the DNN-based SE model to be optimized
in terms of the CER since the DNN mimicking the AM is differentiable.
Consequently, it becomes feasible to build CER-centric SE model that has no
negative effect, e.g., additional calculation cost and changing network
architecture, on the inference phase since our method is merely a training
scheme for the existing DNN-based methods. Experimental results show that our
method improved CER by 8.8% relative derived through a black-box AM although
certain noise levels are kept.
