  Disfluency detection is usually an intermediate step between an automatic
speech recognition (ASR) system and a downstream task. By contrast, this paper
aims to investigate the task of end-to-end speech recognition and disfluency
removal. We specifically explore whether it is possible to train an ASR model
to directly map disfluent speech into fluent transcripts, without relying on a
separate disfluency detection model. We show that end-to-end models do learn to
directly generate fluent transcripts; however, their performance is slightly
worse than a baseline pipeline approach consisting of an ASR system and a
disfluency detection model. We also propose two new metrics that can be used
for evaluating integrated ASR and disfluency models. The findings of this paper
can serve as a benchmark for further research on the task of end-to-end speech
recognition and disfluency removal in the future.
