  In recent years, the remarkable advancements in deep neural networks have
brought tremendous convenience. However, the training process of a highly
effective model necessitates a substantial quantity of samples, which brings
huge potential threats, like unauthorized exploitation with privacy leakage. In
response, we propose a framework named HiddenSpeaker, embedding imperceptible
perturbations within the training speech samples and rendering them unlearnable
for deep-learning-based speaker verification systems that employ large-scale
speakers for efficient training. The HiddenSpeaker utilizes a simplified
error-minimizing method named Single-Level Error-Minimizing (SLEM) to generate
specific and effective perturbations. Additionally, a hybrid objective function
is employed for human perceptual optimization, ensuring the perturbation is
indistinguishable from human listeners. We conduct extensive experiments on
multiple state-of-the-art (SOTA) models in the speaker verification domain to
evaluate HiddenSpeaker. Our results demonstrate that HiddenSpeaker not only
deceives the model with unlearnable samples but also enhances the
imperceptibility of the perturbations, showcasing strong transferability across
different models.
