  Speaker change detection (SCD) is an important feature that improves the
readability of the recognized words from an automatic speech recognition (ASR)
system by breaking the word sequence into paragraphs at speaker change points.
Existing SCD solutions either require additional ensemble for the time based
decisions and recognized word sequences, or implement a tight integration
between ASR and SCD, limiting the potential optimum performance for both tasks.
To address these issues, we propose a novel framework for the SCD task, where
an additional SCD module is built on top of an existing Transformer Transducer
ASR (TT-ASR) network. Two variants of the SCD network are explored in this
framework that naturally estimate speaker change probability for each word,
while allowing the ASR and SCD to have independent optimization scheme for the
best performance. Experiments show that our methods can significantly improve
the F1 score on LibriCSS and Microsoft call center data sets without ASR
degradation, compared with a joint SCD and ASR baseline.
