  Many purely neural network based speech separation approaches have been
proposed to improve objective assessment scores, but they often introduce
nonlinear distortions that are harmful to modern automatic speech recognition
(ASR) systems. Minimum variance distortionless response (MVDR) filters are
often adopted to remove nonlinear distortions, however, conventional neural
mask-based MVDR systems still result in relatively high levels of residual
noise. Moreover, the matrix inverse involved in the MVDR solution is sometimes
numerically unstable during joint training with neural networks. In this study,
we propose a multi-channel multi-frame (MCMF) all deep learning (ADL)-MVDR
approach for target speech separation, which extends our preliminary
multi-channel ADL-MVDR approach. The proposed MCMF ADL-MVDR system addresses
linear and nonlinear distortions. Spatio-temporal cross correlations are also
fully utilized in the proposed approach. The proposed systems are evaluated
using a Mandarin audio-visual corpus and are compared with several
state-of-the-art approaches. Experimental results demonstrate the superiority
of our proposed systems under different scenarios and across several objective
evaluation metrics, including ASR performance.
