  More and more neural network approaches have achieved considerable
improvement upon submodules of speaker diarization system, including speaker
change detection and segment-wise speaker embedding extraction. Still, in the
clustering stage, traditional algorithms like probabilistic linear discriminant
analysis (PLDA) are widely used for scoring the similarity between two speech
segments. In this paper, we propose a supervised method to measure the
similarity matrix between all segments of an audio recording with sequential
bidirectional long short-term memory networks (Bi-LSTM). Spectral clustering is
applied on top of the similarity matrix to further improve the performance.
Experimental results show that our system significantly outperforms the
state-of-the-art methods and achieves a diarization error rate of 6.63% on the
NIST SRE 2000 CALLHOME database.
