  End-to-end speaker diarization approaches have shown exceptional performance
over the traditional modular approaches. To further improve the performance of
the end-to-end speaker diarization for real speech recordings, recently works
have been proposed which integrate unsupervised clustering algorithms with the
end-to-end neural diarization models. However, these methods have a number of
drawbacks: 1) The unsupervised clustering algorithms cannot leverage the
supervision from the available datasets; 2) The K-means-based unsupervised
algorithms that are explored often suffer from the constraint violation
problem; 3) There is unavoidable mismatch between the supervised training and
the unsupervised inference. In this paper, a robust generic neural clustering
approach is proposed that can be integrated with any chunk-level predictor to
accomplish a fully supervised end-to-end speaker diarization model. Also, by
leveraging the sequence modelling ability of a recurrent neural network, the
proposed neural clustering approach can dynamically estimate the number of
speakers during inference. Experimental show that when integrating an
attractor-based chunk-level predictor, the proposed neural clustering approach
can yield better Diarization Error Rate (DER) than the constrained
K-means-based clustering approaches under the mismatched conditions.
