  In modern interactive speech-based systems, speech is consumed and
transcribed incrementally prior to having disfluencies removed. This
post-processing step is crucial for producing clean transcripts and high
performance on downstream tasks (e.g. machine translation). However, most
current state-of-the-art NLP models such as the Transformer operate
non-incrementally, potentially causing unacceptable delays. We propose a
streaming BERT-based sequence tagging model that, combined with a novel
training objective, is capable of detecting disfluencies in real-time while
balancing accuracy and latency. This is accomplished by training the model to
decide whether to immediately output a prediction for the current input or to
wait for further context. Essentially, the model learns to dynamically size its
lookahead window. Our results demonstrate that our model produces comparably
accurate predictions and does so sooner than our baselines, with lower flicker.
Furthermore, the model attains state-of-the-art latency and stability scores
when compared with recent work on incremental disfluency detection.
