  The speaker extraction algorithm extracts the target speech from a mixture
speech containing interference speech and background noise. The extraction
process sometimes over-suppresses the extracted target speech, which not only
creates artifacts during listening but also harms the performance of downstream
automatic speech recognition algorithms. We propose a hybrid continuity loss
function for time-domain speaker extraction algorithms to settle the
over-suppression problem. On top of the waveform-level loss used for superior
signal quality, i.e., SI-SDR, we introduce a multi-resolution delta spectrum
loss in the frequency-domain, to ensure the continuity of an extracted speech
signal, thus alleviating the over-suppression. We examine the hybrid continuity
loss function using a time-domain audio-visual speaker extraction algorithm on
the YouTube LRS2-BBC dataset. Experimental results show that the proposed loss
function reduces the over-suppression and improves the word error rate of
speech recognition on both clean and noisy two-speakers mixtures, without
harming the reconstructed speech quality.
