  In recent years, using raw waveforms as input for deep networks has been
widely explored for the speaker verification system. For example, RawNet and
RawNet2 extracted speaker's feature embeddings from waveforms automatically for
recognizing their voice, which can vastly reduce the front-end computation and
obtain state-of-the-art performance. However, these models do not consider the
speaker's high-level behavioral features, such as intonation, indicating each
speaker's universal style, rhythm, \textit{etc}. This paper presents a novel
network that can handle the intonation information by computing the finite
difference of different speakers' utterance variations. Furthermore, a
hierarchical way is also designed to enhance the intonation property from
coarse to fine to improve the system accuracy. The high-level intonation
features are then fused with the low-level embedding features. Experimental
results on official VoxCeleb1 test data, VoxCeleb1-E, and VoxCeleb-H protocols
show our method outperforms and robustness existing state-of-the-art systems.
To facilitate further research, code is available at
https://github.com/happyjin/FDN
