  In this study we present a Deep Mixture of Experts (DMoE) neural-network
architecture for single microphone speech enhancement. By contrast to most
speech enhancement algorithms that overlook the speech variability mainly
caused by phoneme structure, our framework comprises a set of deep neural
networks (DNNs), each one of which is an 'expert' in enhancing a given speech
type corresponding to a phoneme. A gating DNN determines which expert is
assigned to a given speech segment. A speech presence probability (SPP) is then
obtained as a weighted average of the expert SPP decisions, with the weights
determined by the gating DNN. A soft spectral attenuation, based on the SPP, is
then applied to enhance the noisy speech signal. The experts and the gating
components of the DMoE network are trained jointly. As part of the training,
speech clustering into different subsets is performed in an unsupervised
manner. Therefore, unlike previous methods, a phoneme-labeled database is not
required for the training procedure. A series of experiments with different
noise types verified the applicability of the new algorithm to the task of
speech enhancement. The proposed scheme outperforms other schemes that either
do not consider phoneme structure or use a simpler training methodology.
