  Verifying the identity of a speaker is crucial in modern human-machine
interfaces, e.g., to ensure privacy protection or to enable biometric
authentication. Classical speaker verification (SV) approaches estimate a
fixed-dimensional embedding from a speech utterance that encodes the speaker's
voice characteristics. A speaker is verified if his/her voice embedding is
sufficiently similar to the embedding of the claimed speaker. However, such
approaches assume that only a single speaker exists in the input. The presence
of concurrent speakers is likely to have detrimental effects on the
performance. To address SV in a multi-speaker environment, we propose an
end-to-end deep learning-based SV system that detects whether the target
speaker exists within an input or not. First, an embedding is estimated from a
reference utterance to represent the target's characteristics. Second,
frame-level features are estimated from the input mixture. The reference
embedding is then fused frame-wise with the mixture's features to allow
distinguishing the target from other speakers on a frame basis. Finally, the
fused features are used to predict whether the target speaker is active in the
speech segment or not. Experimental evaluation shows that the proposed method
outperforms the x-vector in multi-speaker conditions.
