  A new learning algorithm for speech separation networks is designed to
explicitly reduce residual noise and artifacts in the separated signal in an
unsupervised manner. Generative adversarial networks are known to be effective
in constructing separation networks when the ground truth for the observed
signal is inaccessible. Still, weak objectives aimed at
distribution-to-distribution mapping make the learning unstable and limit their
performance. This study introduces the remix-cycle-consistency loss as a more
appropriate objective function and uses it to fine-tune adversarially learned
source separation models. The remix-cycle-consistency loss is defined as the
difference between the mixed speech observed at microphones and the
pseudo-mixed speech obtained by alternating the process of separating the mixed
sound and remixing its outputs with another combination. The minimization of
this loss leads to an explicit reduction in the distortions in the output of
the separation network. Experimental comparisons with multichannel speech
separation demonstrated that the proposed method achieved high separation
accuracy and learning stability comparable to supervised learning.
