  Speaker verification (SV) has recently attracted considerable research
interest due to the growing popularity of virtual assistants. At the same time,
there is an increasing requirement for an SV system: it should be robust to
short speech segments, especially in noisy and reverberant environments. In
this paper, we consider one more important requirement for practical
applications: the system should be robust to an audio stream containing long
non-speech segments, where a voice activity detection (VAD) is not applied. To
meet these two requirements, we introduce feature pyramid module (FPM)-based
multi-scale aggregation (MSA) and self-adaptive soft VAD (SAS-VAD). We present
the FPM-based MSA to deal with short speech segments in noisy and reverberant
environments. Also, we use the SAS-VAD to increase the robustness to long
non-speech segments. To further improve the robustness to acoustic distortions
(i.e., noise and reverberation), we apply a masking-based speech enhancement
(SE) method. We combine SV, VAD, and SE models in a unified deep learning
framework and jointly train the entire network in an end-to-end manner. To the
best of our knowledge, this is the first work combining these three models in a
deep learning framework. We conduct experiments on Korean indoor (KID) and
VoxCeleb datasets, which are corrupted by noise and reverberation. The results
show that the proposed method is effective for SV in the challenging conditions
and performs better than the baseline i-vector and deep speaker embedding
systems.
