  We propose a new end-to-end neural diarization (EEND) system that is based on
Conformer, a recently proposed neural architecture that combines convolutional
mappings and Transformer to model both local and global dependencies in speech.
We first show that data augmentation and convolutional subsampling layers
enhance the original self-attentive EEND in the Transformer-based EEND, and
then Conformer gives an additional gain over the Transformer-based EEND.
However, we notice that the Conformer-based EEND does not generalize as well
from simulated to real conversation data as the Transformer-based model. This
leads us to quantify the mismatch between simulated data and real speaker
behavior in terms of temporal statistics reflecting turn-taking between
speakers, and investigate its correlation with diarization error. By mixing
simulated and real data in EEND training, we mitigate the mismatch further,
with Conformer-based EEND achieving 24% error reduction over the baseline
SA-EEND system, and 10% improvement over the best augmented Transformer-based
system, on two-speaker CALLHOME data.
