  Automatic speech recognition (ASR) models make fewer errors when more
surrounding speech information is presented as context. Unfortunately,
acquiring a larger future context leads to higher latency. There exists an
inevitable trade-off between speed and accuracy. Naively, to fit different
latency requirements, people have to store multiple models and pick the best
one under the constraints. Instead, a more desirable approach is to have a
single model that can dynamically adjust its latency based on different
constraints, which we refer to as Multi-mode ASR. A Multi-mode ASR model can
fulfill various latency requirements during inference -- when a larger latency
becomes acceptable, the model can process longer future context to achieve
higher accuracy and when a latency budget is not flexible, the model can be
less dependent on future context but still achieve reliable accuracy. In
pursuit of Multi-mode ASR, we propose Stochastic Future Context, a simple
training procedure that samples one streaming configuration in each iteration.
Through extensive experiments on AISHELL-1 and LibriSpeech datasets, we show
that a Multi-mode ASR model rivals, if not surpasses, a set of competitive
streaming baselines trained with different latency budgets.
