  The increasing reliability of automatic speech recognition has proliferated
its everyday use. However, for research purposes, it is often unclear which
model one should choose for a task, particularly if there is a requirement for
speed as well as accuracy. In this paper, we systematically evaluate six speech
recognizers using metrics including word error rate, latency, and the number of
updates to already recognized words on English test data, as well as propose
and compare two methods for streaming audio into recognizers for incremental
recognition. We further propose Revokes per Second as a new metric for
evaluating incremental recognition and demonstrate that it provides insights
into overall model performance. We find that, generally, local recognizers are
faster and require fewer updates than cloud-based recognizers. Finally, we find
Meta's Wav2Vec model to be the fastest, and find Mozilla's DeepSpeech model to
be the most stable in its predictions.
