  Although the conventional mask-based minimum variance distortionless response
(MVDR) could reduce the non-linear distortion, the residual noise level of the
MVDR separated speech is still high. In this paper, we propose a
spatio-temporal recurrent neural network based beamformer (RNN-BF) for target
speech separation. This new beamforming framework directly learns the
beamforming weights from the estimated speech and noise spatial covariance
matrices. Leveraging on the temporal modeling capability of RNNs, the RNN-BF
could automatically accumulate the statistics of the speech and noise
covariance matrices to learn the frame-level beamforming weights in a recursive
way. An RNN-based generalized eigenvalue (RNN-GEV) beamformer and a more
generalized RNN beamformer (GRNN-BF) are proposed. We further improve the
RNN-GEV and the GRNN-BF by using layer normalization to replace the commonly
used mask normalization on the covariance matrices. The proposed GRNN-BF
obtains better performance against prior arts in terms of speech quality
(PESQ), speech-to-noise ratio (SNR) and word error rate (WER).
