  This work addresses the problem of multichannel source separation combining
two powerful approaches, multichannel spectral factorization with recent
monophonic deep-learning (DL) based spectrum inference. Individual source
spectra at different channels are estimated with a Masker-Denoiser Twin Network
(MaD TwinNet), able to model long-term temporal patterns of a musical piece.
The monophonic source spectrograms are used within a spatial covariance mixing
model based on Complex Non-Negative Matrix Factorization (CNMF) that predicts
the spatial characteristics of each source. The proposed framework is evaluated
on the task of singing voice separation with a large multichannel dataset.
Experimental results show that our joint DL+CNMF method outperforms both the
individual monophonic DL-based separation and the multichannel CNMF baseline
methods.
